Enhanced Multivariate Time Series Analysis Using LSTM: A Comparative Study of Min-Max and Z-Score Normalization Techniques


Andri Pranolo(1*); Faradini Usha Setyaputri(2); Andien Khansa’a Iffat Paramarta(3); Alfiansyah Putra Pertama Triono(4); Akhmad Fanny Fadhilla(5); Ade Kurnia Ganesh Akbari(6); Agung Bella Putra Utama(7); Aji Prasetya Wibawa(8); Wako Uriu(9);

(1) Universitas Ahmad Dahlan
(2) Universitas Negeri Malang
(3) Universitas Negeri Malang
(4) Universitas Negeri Malang
(5) Universitas Negeri Malang
(6) Universitas Negeri Malang
(7) Universitas Negeri Malang
(8) Universitas Negeri Malang
(9) Chikushi Jogakuen University
(*) Corresponding Author

  

Abstract


The primary objective of this study is to analyze multivariate time series data by employing the Long Short-Term Memory (LSTM) model. Deep learning models often face issues when dealing with multivariate time series data, which is defined by several variables that have diverse value ranges. These challenges arise owing to the potential biases present in the data. In order to tackle this issue, it is crucial to employ normalization techniques such as min-max and z-score to guarantee that the qualities are standardized and can be compared effectively. This study assesses the effectiveness of the LSTM model by applying two normalizing techniques in five distinct attribute selection scenarios. The aim of this study is to ascertain the normalization strategy that produces the most precise outcomes when employed in the LSTM model for the analysis of multivariate time series. The evaluation measures employed in this study comprise Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE), and R-Squared (R2). The results suggest that the min-max normalization method regularly yields superior outcomes in comparison to the z-score method. Min-max normalization specifically resulted in a decreased mean absolute percentage error (MAPE) and root mean square error (RMSE), as well as an increased R-squared (R2) value. These improvements indicate enhanced accuracy and performance of the model. This paper makes a significant contribution by doing a thorough comparison analysis of normalizing procedures. It offers vital insights for researchers and practitioners in choosing suitable preprocessing strategies to improve the performance of deep learning models. The study's findings underscore the importance of selecting the appropriate normalization strategy to enhance the precision and dependability of multivariate time series predictions using LSTM models. To summarize, the results indicate that min-max normalization is superior to z-score normalization for this particular use case. This provides a useful suggestion for further studies and practical applications in the field. This study emphasizes the significance of normalization in analyzing multivariate time series and contributes to the larger comprehension of data preprocessing in deep learning models

Keywords


Multivariate time series; LSTM; normalization; min-max; z-score

  
  

Full Text:

PDF
  

Article Metrics

Abstract view: 83 times
PDF view: 25 times 		     

Digital Object Identifier

doi  https://doi.org/10.33096/ilkom.v16i2.2333.210-220 		  

Cite

How to cite item

References


Y. Zou, R. V. Donner, N. Marwan, J. F. Donges, and J. Kurths, “Complex network approaches to nonlinear time series analysis,” Phys. Rep., vol. 787, pp. 1–97, Jan. 2019, doi: 10.1016/j.physrep.2018.10.005.

F. Karim, S. Majumdar, H. Darabi, and S. Harford, “Multivariate LSTM-FCNs for time series classification,” Neural Networks, vol. 116, pp. 237–245, Aug. 2019, doi: 10.1016/j.neunet.2019.04.014.

C. Qin, M. Liu, X. Guo, and J. Liu, “Human Resources in Primary Healthcare Institutions before and after the New Healthcare Reform in China from 2003 to 2019: An Interrupted Time Series Analysis,” Int. J. Environ. Res. Public Health, vol. 19, no. 10, 2022, doi: 10.3390/ijerph19106042.

H. Apaydin, H. Feizi, M. T. Sattari, M. S. Colak, S. Shamshirband, and K. W. Chau, “Comparative analysis of recurrent neural network architectures for reservoir inflow forecasting,” Water (Switzerland), vol. 12, no. 5, pp. 1–18, 2020, doi: 10.3390/w12051500.

A. B. P. Utama, A. P. Wibawa, Muladi, and A. Nafalski, “PSO based Hyperparameter tuning of CNN Multivariate Time-Series Analysis,” J. Online Inform., vol. 7, no. 2, pp. 193–202, 2022, doi: 10.15575/join.v7i2.858.

J. Runge and R. Zmeureanu, “Forecasting Energy Use in Buildings Using Artificial Neural Networks: A Review,” Energies, vol. 12, no. 17, p. 3254, Aug. 2019, doi: 10.3390/en12173254.

H. Hewamalage, C. Bergmeir, and K. Bandara, “Recurrent Neural Networks for Time Series Forecasting: Current status and future directions,” Int. J. Forecast., vol. 37, no. 1, pp. 388–427, Jan. 2021, doi: 10.1016/j.ijforecast.2020.06.008.

I. Koprinska, D. Wu, and Z. Wang, “Convolutional Neural Networks for Energy Time Series Forecasting,” in 2018 International Joint Conference on Neural Networks (IJCNN), Jul. 2018, pp. 1–8, doi: 10.1109/IJCNN.2018.8489399.

G. Bathla, R. Rani, and H. Aggarwal, “Stocks of year 2020: prediction of high variations in stock prices using LSTM,” Multimed. Tools Appl., vol. 82, no. 7, pp. 9727–9743, Mar. 2023, doi: 10.1007/s11042-022-12390-5.

A. N. . F. Faisal, A. Rahman, M. T. M. Habib, A. H. Siddique, M. Hasan, and M. M. Khan, “Neural networks based multivariate time series forecasting of solar radiation using meteorological data of different cities of Bangladesh,” Results Eng., vol. 13, p. 100365, Mar. 2022, doi: 10.1016/j.rineng.2022.100365.

M. Yang and J. Wang, “Adaptability of Financial Time Series Prediction Based on BiLSTM,” Procedia Comput. Sci., vol. 199, pp. 18–25, 2022, doi: 10.1016/j.procs.2022.01.003.

O. Ben Fredj, A. Mihoub, M. Krichen, O. Cheikhrouhou, and A. Derhab, “CyberSecurity Attack Prediction: A Deep Learning Approach,” in 13th International Conference on Security of Information and Networks, Nov. 2020, pp. 1–6, doi: 10.1145/3433174.3433614.

T. A. Rashid, P. Fattah, and D. K. Awla, “Using Accuracy Measure for Improving the Training of LSTM with Metaheuristic Algorithms,” Procedia Comput. Sci., vol. 140, pp. 324–333, 2018, doi: 10.1016/j.procs.2018.10.307.

N. Passalis, A. Tefas, J. Kanniainen, M. Gabbouj, and A. Iosifidis, “Deep Adaptive Input Normalization for Time Series Forecasting,” IEEE Trans. Neural Networks Learn. Syst., vol. 31, no. 9, pp. 3760–3765, Sep. 2020, doi: 10.1109/TNNLS.2019.2944933.

J. Wang, G. Wen, S. Yang, and Y. Liu, “Remaining Useful Life Estimation in Prognostics Using Deep Bidirectional LSTM Neural Network,” in 2018 Prognostics and System Health Management Conference (PHM-Chongqing), Oct. 2018, pp. 1037–1042, doi: 10.1109/PHM-Chongqing.2018.00184.

A. P. Wibawa, W. Lestari, A. B. P. Utama, I. T. Saputra, and Z. N. Izdihar, “Multilayer Perceptron untuk Prediksi Sessions pada Sebuah Website Journal Elektronik,” Indones. J. Data Sci., vol. 1, no. 3, Dec. 2020, doi: 10.33096/ijodas.v1i3.15.

A. P. Wibawa, Z. N. Izdihar, A. B. P. Utama, L. Hernandez, and Haviluddin, “Min-Max Backpropagation Neural Network to Forecast e-Journal Visitors,” in 2021 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Apr. 2021, pp. 052–058, doi: 10.1109/ICAIIC51459.2021.9415197.

A. P. Wibawa, “Mean-Median Smoothing Backpropagation Neural Network to Forecast Unique Visitors Time Series of Electronic Journal,” J. Appl. Data Sci., vol. 4, no. 3, pp. 163–174, Sep. 2023, doi: 10.47738/jads.v4i3.97.

A. P. Wibawa, I. T. Saputra, A. B. P. Utama, W. Lestari, and Z. N. Izdihar, “Long Short-Term Memory to Predict Unique Visitors of an Electronic Journal,” in 2020 6th International Conference on Science in Information Technology (ICSITech), Oct. 2020, pp. 176–179, doi: 10.1109/ICSITech49800.2020.9392031.

A. P. Wibawa, R. R. Ula, A. B. P. Utama, M. Y. Chuttur, A. Pranolo, and Haviluddin, “Forecasting e-Journal Unique Visitors using Smoothed Long Short-Term Memory,” in 2021 7th International Conference on Electrical, Electronics and Information Engineering (ICEEIE), Oct. 2021, pp. 609–613, doi: 10.1109/ICEEIE52663.2021.9616628.

W. Y. Ayele, “Adapting CRISP-DM for Idea Mining,” Int. J. Adv. Comput. Sci. Appl., vol. 11, no. 6, 2020, doi: 10.14569/IJACSA.2020.0110603.

H. Wiemer, L. Drowatzky, and S. Ihlenfeldt, “Data Mining Methodology for Engineering Applications (DMME)—A Holistic Extension to the CRISP-DM Model,” Appl. Sci., vol. 9, no. 12, p. 2407, Jun. 2019, doi: 10.3390/app9122407.

C. Schröer, F. Kruse, and J. M. Gómez, “A systematic literature review on applying CRISP-DM process model,” Procedia Comput. Sci., vol. 181, no. 2019, pp. 526–534, 2021, doi: 10.1016/j.procs.2021.01.199.

X. Xiao, J. Liu, D. Liu, Y. Tang, and F. Zhang, “Condition Monitoring of Wind Turbine Main Bearing Based on Multivariate Time Series Forecasting,” Energies, vol. 15, no. 5, p. 1951, Mar. 2022, doi: 10.3390/en15051951.

S. Bhanja and A. Das, “Deep Neural Network for Multivariate Time-Series Forecasting,” 2021, pp. 267–277.

S. Huber, H. Wiemer, D. Schneider, and S. Ihlenfeldt, “DMME: Data mining methodology for engineering applications – a holistic extension to the CRISP-DM model,” Procedia CIRP, vol. 79, pp. 403–408, 2019, doi: 10.1016/j.procir.2019.02.106.

T. Emmanuel, T. Maupong, D. Mpoeleng, T. Semong, B. Mphago, and O. Tabona, “A survey on missing data in machine learning,” J. Big Data, vol. 8, no. 1, p. 140, Oct. 2021, doi: 10.1186/s40537-021-00516-9.

A. Mirzaei, S. R. Carter, A. E. Patanwala, and C. R. Schneider, “Missing data in surveys: Key concepts, approaches, and applications,” Res. Soc. Adm. Pharm., vol. 18, no. 2, pp. 2308–2316, Feb. 2022, doi: 10.1016/j.sapharm.2021.03.009.

P. J. Muhammad Ali, “Investigating the Impact of Min-Max Data Normalization on the Regression Performance of K-Nearest Neighbor with Different Similarity Measurements,” Aro-the Sci. J. Koya Univ., vol. 10, no. 1, pp. 85–91, 2022, doi: 10.14500/aro.10955.

H. Henderi, “Comparison of Min-Max normalization and Z-Score Normalization in the K-nearest neighbor (kNN) Algorithm to Test the Accuracy of Types of Breast Cancer,” IJIIS Int. J. Informatics Inf. Syst., vol. 4, no. 1, pp. 13–20, 2021, doi: 10.47738/ijiis.v4i1.73.

A. J. Mohammed, “Improving Classification Performance for a Novel Imbalanced Medical Dataset using SMOTE Method,” Int. J. Adv. Trends Comput. Sci. Eng., vol. 9, no. 3, pp. 3161–3172, Jun. 2020, doi: 10.30534/ijatcse/2020/104932020.

S. F. M. Radzi, M. K. A. Karim, M. I. Saripan, M. A. A. Rahman, I. N. C. Isa, and M. J. Ibahim, “Hyperparameter tuning and pipeline optimization via grid search method and tree-based autoML in breast cancer prediction,” J. Pers. Med., vol. 11, no. 10, 2021, doi: 10.3390/jpm11100978.

B. H. Shekar and G. Dagnew, “Grid Search-Based Hyperparameter Tuning and Classification of Microarray Cancer Data,” in 2019 Second International Conference on Advanced Computational and Communication Paradigms (ICACCP), Feb. 2019, pp. 1–8, doi: 10.1109/ICACCP.2019.8882943.

A. Pranolo, Y. Mao, A. P. Wibawa, A. B. P. Utama, and F. A. Dwiyanto, “Robust LSTM With Tuned-PSO and Bifold-Attention Mechanism for Analyzing Multivariate Time-Series,” IEEE Access, vol. 10, pp. 78423–78434, 2022, doi: 10.1109/ACCESS.2022.3193643.

W. Elmasry, A. Akbulut, and A. H. Zaim, “Evolving deep learning architectures for network intrusion detection using a double PSO metaheuristic,” Comput. Networks, vol. 168, p. 107042, Feb. 2020, doi: 10.1016/j.comnet.2019.107042.

W. Sun and C. Huang, “A novel carbon price prediction model combines the secondary decomposition algorithm and the long short-term memory network,” Energy, vol. 207, p. 118294, Sep. 2020, doi: 10.1016/j.energy.2020.118294.

W. M. Ridwan, M. Sapitang, A. Aziz, K. F. Kushiar, A. N. Ahmed, and A. El-Shafie, “Rainfall forecasting model using machine learning methods: Case study Terengganu, Malaysia,” Ain Shams Eng. J., vol. 12, no. 2, pp. 1651–1663, Jun. 2021, doi: 10.1016/j.asej.2020.09.011.

T. B. Pun and T. B. Shahi, “Nepal Stock Exchange Prediction Using Support Vector Regression and Neural Networks,” Proc. 2018 2nd Int. Conf. Adv. Electron. Comput. Commun. ICAECC 2018, pp. 1–6, 2018, doi: 10.1109/ICAECC.2018.8479456.

Z. Lyu et al., Neuroevolution of recurrent neural networks for time series forecasting of coal-fired power plant operating parameters, vol. 1, no. 1. Association for Computing Machinery, 2021.


Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Andri Pranolo, Faradini Usha Setyaputri, Andien Khansa’a Iffat Paramarta, Alfiansyah Putra Pertama Triono, Akhmad Fanny Fadhilla, Ade Kurnia Ganesh Akbari, Agung Bella Putra Utama, Aji Prasetya Wibawa, Wako Uriu

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.