Vol. 3 No. 1 (2023): Issue 3
Articles

Residual Self-Attention-Based Temporal Deep Model for Predicting Aircraft Engine Failure within a Specific Cycle

PDF
  • Tong Zhou,
  • Guojun Zhang ,
  • Yiqun Cai
Tong Zhou
Air China Cargo Co., Ltd., Beijing, 101318, CHINA
Guojun Zhang
Quadrant international inc., San Diego, 92121, USA
Yiqun Cai
University of Florida, Herbert Wertheim College, FL, 32608, USA
DOI: https://doi.org/10.71070/oaml.v3i1.75

Published 2023-10-15

Keywords

  • Component,
  • Aircraft Engine Failure Prediction,
  • Self-attention,
  • Residual Connection

How to Cite

Zhou, T., Zhang , G., & Cai, Y. (2023). Residual Self-Attention-Based Temporal Deep Model for Predicting Aircraft Engine Failure within a Specific Cycle. Optimizations in Applied Machine Learning, 3(1). https://doi.org/10.71070/oaml.v3i1.75
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Abstract

Aircraft engines are complex, high-performance machines operating under extreme conditions, where reliability is critical for aviation safety. Early detection of engine faults is essential not only to ensure passenger safety but also to reduce operational costs and maintain efficient flight schedules. Traditional fault detection methods rely on rule-based diagnostics, setting predetermined thresholds for engine parameters such as temperature and pressure. However, these methods are limited in accuracy and often fail to capture the intricate degradation patterns of engine components, leading to late fault detection and frequent false alarms. Recent advancements in machine learning, particularly in deep learning, offer promising alternatives. Machine learning models can analyze large-scale time-series data and recognize complex patterns that human expertise might overlook. Among these, Long Short-Term Memory (LSTM) networks, combined with self-attention mechanisms, have shown potential in capturing temporal dependencies crucial for predictive maintenance. This study proposes a Residual Self-Attention-based LSTM model for predicting aircraft engine failures. By integrating residual connections with self-attention, the model enhances pattern recognition and interpretability, offering improved fault prediction accuracy over traditional models. The model was trained on aircraft engine sensor data, achieving high performance across various metrics, suggesting that this architecture holds significant promise for proactive aircraft maintenance applications.

PDF

References

  1. Kerrebrock, J. L. (1992). Aircraft engines and gas turbines. MIT press.
  2. Pollock, T. M. (2016). Alloy design for aircraft engines. Nature materials, 15(8), 809-815.
  3. Wenjun, D., Fatahizadeh, M., Touchaei, H. G., Moayedi, H., & Foong, L. K. (2023). Application of six neural network-based solutions on bearing capacity of shallow footing on double-layer soils. Steel and Composite Structures, 49(2), 231-244.
  4. Dai, W. (2023). Design of traffic improvement plan for line 1 Baijiahu station of Nanjing metro. Innovations in Applied Engineering and Technology.
  5. Ren, P., Zhao, Z., & Yang, Q. (2023). Exploring the Path of Transformation and Development for Study Abroad Consultancy Firms in China. Journal of Computational Methods in Engineering Applications, 1-12.
  6. Zhao, Z., Ren, P., & Yang, Q. (2023). Student Self-Management, Academic Achievement: Exploring the Mediating Role of Self-Efficacy and the Moderating Influence of Gender—Insights From a Survey Conducted in 3 Universities in America. Journal of Integrated Social Sciences and Humanities, 1-12.
  7. Dai, W. (2022). Evaluation and improvement of carrying capacity of a traffic system. Innovations in Applied Engineering and Technology, 1-9.
  8. Li, C., & Tang, Y. (2023). The Factors of Brand Reputation in Chinese Luxury Fashion Brands. Journal of Integrated Social Sciences and Humanities, 1-14.
  9. Dai, W. (2021). Safety evaluation of traffic system with historical data based on Markov process and deep-reinforcement learning. Journal of Computational Methods in Engineering Applications, 1-14.
  10. Xiong, S., Zhang, H., Wang, M., & Zhou, N. (2022). Distributed Data Parallel Acceleration-Based Generative Adversarial Network for Fingerprint Generation. Innovations in Applied Engineering and Technology, 1-12.
  11. Xiong, S., Chen, X., & Zhang, H. (2023). Deep Learning-Based Multifunctional End-to-End Model for Optical Character Classification and Denoising. Journal of Computational Methods in Engineering Applications, 1-13.
  12. Xiong, S., Zhang, H., & Wang, M. (2022). Ensemble Model of Attention Mechanism-Based DCGAN and Autoencoder for Noised OCR Classification. Journal of Electronic & Information Systems, 4(1), 33-41.
  13. Yu, L., Li, J., Cheng, S., Xiong, S., & Shen, H. (2013). Secure continuous aggregation in wireless sensor networks. IEEE Transactions on Parallel and Distributed Systems, 25(3), 762-774.
  14. Xiong, S., Yu, L., Shen, H., Wang, C., & Lu, W. (2012, March). Efficient algorithms for sensor deployment and routing in sensor networks for network-structured environment monitoring. In 2012 Proceedings IEEE INFOCOM (pp. 1008-1016). IEEE.
  15. Feng, Z., Xiong, S., Cao, D., Deng, X., Wang, X., Yang, Y., ... & Wu, G. (2015, March). Hrs: A hybrid framework for malware detection. In Proceedings of the 2015 ACM International Workshop on International Workshop on Security and Privacy Analytics (pp. 19-26).
  16. Xiong, S., Li, J., Li, M., Wang, J., & Liu, Y. (2011, April). Multiple task scheduling for low-duty-cycled wireless sensor networks. In 2011 Proceedings IEEE INFOCOM (pp. 1323-1331). IEEE.
  17. Yu, L., Li, J., Cheng, S., & Xiong, S. (2011, April). Secure continuous aggregation via sampling-based verification in wireless sensor networks. In 2011 Proceedings IEEE INFOCOM (pp. 1763-1771). IEEE.
  18. Chen, X., & Zhang, H. (2023). Performance Enhancement of AlGaN-based Deep Ultraviolet Light-emitting Diodes with AlxGa1-xN Linear Descending Layers. Innovations in Applied Engineering and Technology, 1-10.
  19. Zhou, Z., Wu, J., Cao, Z., She, Z., Ma, J., & Zu, X. (2021, September). On-Demand Trajectory Prediction Based on Adaptive Interaction Car Following Model with Decreasing Tolerance. In 2021 International Conference on Computers and Automation (CompAuto) (pp. 67-72). IEEE.
  20. Wang, H., Li, J., & Xiong, S. (2008). Efficient join algorithms for distributed information integration based on XML. International Journal of Business Process Integration and Management, 3(4), 271-281.
  21. Xiong, S., & Li, J. (2009, April). Optimizing many-to-many data aggregation in wireless sensor networks. In Asia-Pacific Web Conference (pp. 550-555). Berlin, Heidelberg: Springer Berlin Heidelberg.
  22. Medsker, L. R., & Jain, L. (2001). Recurrent neural networks. Design and Applications, 5(64-67), 2.
  23. Grossberg, S. (2013). Recurrent neural networks. Scholarpedia, 8(2), 1888.
  24. Salehinejad, H., Sankar, S., Barfett, J., Colak, E., & Valaee, S. (2017). Recent advances in recurrent neural networks. arXiv preprint arXiv:1801.01078.
  25. Graves, A., & Graves, A. (2012). Long short-term memory. Supervised sequence labelling with recurrent neural networks, 37-45.
  26. Van Houdt, G., Mosquera, C., & Nápoles, G. (2020). A review on the long short-term memory model. Artificial Intelligence Review, 53(8), 5929-5955.
  27. Vakharia, V., Gupta, V. K., & Kankar, P. K. (2015). Ball bearing fault diagnosis using supervised and unsupervised machine learning methods. Int J Acoust Vib, 20(4), 244-250.
  28. Bandt, C., & Pompe, B. (2002). Permutation entropy: a natural complexity measure for time series. Physical review letters, 88(17), 174102.
  29. Chen, Z., Cao, S., & Mao, Z. (2017). Remaining useful life estimation of aircraft engines using a modified similarity and supporting vector machine (SVM) approach. Energies, 11(1), 28.
  30. Abdi, H., & Williams, L. J. (2010). Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4), 433-459.
  31. Jolliffe, I. (2005). Principal component analysis. Encyclopedia of statistics in behavioral science.
  32. Shlens, J. (2014). A tutorial on principal component analysis. arXiv preprint arXiv:1404.1100.
  33. Laghrissi, F., Douzi, S., Douzi, K., & Hssina, B. (2021). Intrusion detection systems using long short-term memory (LSTM). Journal of Big Data, 8(1), 65.
  34. Kratzert, F., Klotz, D., Brenner, C., Schulz, K., & Herrnegger, M. (2018). Rainfall–runoff modelling using long short-term memory (LSTM) networks. Hydrology and Earth System Sciences, 22(11), 6005-6022.
  35. Huang, R., Wei, C., Wang, B., Yang, J., Xu, X., Wu, S., & Huang, S. (2022). Well performance prediction based on Long Short-Term Memory (LSTM) neural network. Journal of Petroleum Science and Engineering, 208, 109686.
  36. Shaw, P., Uszkoreit, J., & Vaswani, A. (2018). Self-attention with relative position representations. arXiv preprint arXiv:1803.02155.
  37. Zhao, H., Jia, J., & Koltun, V. (2020). Exploring self-attention for image recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10076-10085).
  38. Zhang, H., Goodfellow, I., Metaxas, D., & Odena, A. (2019, May). Self-attention generative adversarial networks. In International conference on machine learning (pp. 7354-7363). PMLR.
  39. Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017, February). Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the AAAI conference on artificial intelligence (Vol. 31, No. 1).
  40. Xie, S., Zhang, H., Guo, J., Tan, X., Bian, J., Awadalla, H. H., ... & Yan, R. (2023). Residual: Transformer with dual residual connections. arXiv preprint arXiv:2304.14802.
  41. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
  42. Banerjee, K., Gupta, R. R., Vyas, K., & Mishra, B. (2020). Exploring alternatives to softmax function. arXiv preprint arXiv:2011.11538.
  43. Lau, M. M., & Lim, K. H. (2018, December). Review of adaptive activation function in deep neural network. In 2018 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES) (pp. 686-690). IEEE.