Deepfake Detection in Videos Using Long Short-Term Memory and CNN ResNext


Muhammad Indra Abidin(1*); Ingrid Nurtanio(2); Andani Achmad(3);

(1) Universitas Hasanuddin
(2) Universitas Hasanuddin
(3) Universitas Hasanuddin
(*) Corresponding Author

  

Abstract


Deep-fake in videos is a video synthesis technique by changing the people’s face in the video with others’ face. Deep-fake technology in videos has been used to manipulate information, therefore it is necessary to detect deep-fakes in videos. This paper aimed to detect deep-fakes in videos using the ResNext Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) algorithms. The video data was divided into 4 types, namely video with 10 frames, 20 frames, 40 frames and 60 frames. Furthermore, face detection was used to crop the image to 100 x 100 pixels and then the pictures were processed using ResNext CNN and LSTM. The confusion matrix was employed to measure the performance of the ResNext CNN-LSTM algorithm. The indicators used were accuracy, precision, and recall. The results of data classification showed that the highest accuracy value was 90% for data with 40 and 60 frames. While data with 10 frames had the lowest accuracy with 52% only. ResNext CNN-LSTM was able to detect deep-fakes in videos well even though the size of the image was small.

Keywords


Deepfake; ResNext CNN; LSTM

  
  

Full Text:

PDF
  

Article Metrics

Abstract view: 1233 times
PDF view: 361 times 		     

Digital Object Identifier

doi  https://doi.org/10.33096/ilkom.v14i3.1254.178-185 		  

Cite

How to cite item

References


A. Brunetti, D. Buongiorno, G. F. Trotta, and V. Bevilacqua, “Computer vision and deep learning techniques for pedestrian detection and tracking: A survey,” Neurocomputing, vol. 300, pp. 17–33, 2018.

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-Decem, pp. 2818–2826, 2016.

H. Wang, Y. Wang, Z. Zhou, X. Ji, D. Gong, and J. Zhou, “CosFace: Large Margin Cosine Loss for Deep Face Recognition,” Cvpr, pp. 5265–5274, 2018.

O. Victoria and I. P. Solihin, “Pendeteksi wajah secara realtime menggunakan metode Eigenface,” SEINASI-KESI (Seminar Nas. Inform. Sist. Inf. Dan Keamanan Siber), pp. 126–131, 2018.

D. Pan, L. Sun, R. Wang, X. Zhang, and R. O. Sinnott, “Deepfake Detection through Deep Learning,” Proc. - 2020 IEEE/ACM Int. Conf. Big Data Comput. Appl. Technol. BDCAT 2020, pp. 134–143, 2020.

M. Westerlund, “The emergence of deepfake technology: A review,” Technol. Innov. Manag. Rev., vol. 9, no. 11, pp. 39–52, 2019.

S. T. Suganthi et al., “Deep learning model for deep fake face recognition and detection,” PeerJ Comput. Sci., vol. 8, pp. 1–20, 2022.

S. Agarwal, H. Farid, T. El-Gaaly, and S. N. Lim, “Detecting Deep-Fake Videos from Appearance and Behavior,” 2020 IEEE Int. Work. Inf. Forensics Secur. WIFS 2020, 2020.

S. Megawan, W. S. Lestari, and A. Halim, “Deteksi non-spoofing wajah pada video secara real time menggunakan Faster R-CNN,” vol. 3, no. 3, 2022.

P. Ranjan, S. Patil, and F. Kazi, “Improved generalizability of deep-fakes detection using transfer learning based CNN framework,” Proc. - 3rd Int. Conf. Inf. Comput. Technol. ICICT 2020, pp. 86–90, 2020.

I. A. Anjani, Y. R. Pratiwi, and S. Norfa Bagas Nurhuda, “Implementation of Deep Learning using Convolutional Neural Network Algorithm for Classification Rose Flower,” J. Phys. Conf. Ser., vol. 1842, no. 1, 2021.

Q. Zhang, M. Zhang, T. Chen, Z. Sun, Y. Ma, and B. Yu, “Recent advances in convolutional neural network acceleration,” Neurocomputing, vol. 323, pp. 37–51, 2019.

M. A. Saleem, N. Senan, F. Wahid, M. Aamir, A. Samad, and M. Khan, “Comparative analysis of recent architecture of Convolutional Neural Network,” Math. Probl. Eng., vol. 2022, 2022.

K. Annapurani and D. Ravilla, “CNN based image classification model,” Int. J. Innov. Technol. Explor. Eng., vol. 8, no. 11 Special Issue, pp. 1106–1114, 2019.

S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 5987–5995, 2017.

G. Pant, D. P. Yadav, and A. Gaur, “ResNeXt convolution neural network topology-based deep learning model for identification and classification of Pediastrum,” Algal Res., vol. 48, no. April, p. 101932, 2020.

P. N. Srinivasu, J. G. Sivasai, M. F. Ijaz, A. K. Bhoi, W. Kim, and J. J. Kang, “Classification of skin disease using deep learning neural networks with mobilenet v2 and lstm,” Sensors, vol. 21, no. 8, pp. 1–27, 2021.

C. I. Garcia, F. Grasso, A. Luchetta, M. C. Piccirilli, L. Paolucci, and G. Talluri, “A comparison of power quality disturbance detection and classification methods using CNN, LSTM and CNN-LSTM,” Appl. Sci., vol. 10, no. 19, pp. 1–22, 2020.

X. Li, S. Li, J. Li, J. Yao, and X. Xiao, “Detection of fake-video uploaders on social media using Naive Bayesian model with social cues,” Sci. Rep., vol. 11, no. 1, pp. 1–11, 2021.

O. Caelen, “A Bayesian interpretation of the confusion matrix,” Ann. Math. Artif. Intell., vol. 81, no. 3–4, pp. 429–450, 2017.


Refbacks

  • There are currently no refbacks.


Copyright (c) 2022 Muhammad Indra Abidin, Ingrid Nurtanio, Andani Achmad

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.