Vol. 5 No. 1 (2025): Issue 5
Articles

Predictive Energy Management Strategy for Hybrid Electric Vehicles Based on Soft Actor-Critic

PDF
  • Weidong Huang,
  • Jiahuai Ma
Weidong Huang
Upower Energy Technology (Guang Zhou) Co..Ltd., Guangzhou, 510000, CHINA
Jiahuai Ma
Corresponding, University of Florida, Herbert Wertheim College, FL, 32608, USA
DOI: https://doi.org/10.71070/es.v5i1.82

Published 2025-01-25

Keywords

  • Hybrid electric vehicles,
  • Energy management,
  • Vehicle speed prediction,
  • Reinforcement learning

How to Cite

Huang, W., & Ma, J. (2025). Predictive Energy Management Strategy for Hybrid Electric Vehicles Based on Soft Actor-Critic. Energy & System, 5(1). https://doi.org/10.71070/es.v5i1.82

Abstract

This paper presents a predictive Energy Management Strategy (EMS) for series hybrid electric vehicles based on an improved Soft Actor-Critic (SAC) algorithm. First, the Informer model is used to predict the vehicle's short-term speed trajectory, providing foresight to guide the optimization of the energy management strategy. Second, by incorporating a prioritized experience replay strategy, the convergence of the SAC algorithm is accelerated, and its performance is enhanced. Finally, a simulation environment based on real driving cycles was constructed, and the simulation results demonstrate that our method effectively reduces fuel consumption, achieving approximately a 6.1% performance improvement over the original SAC algorithm. This not only validates the superiority of our approach over traditional methods in terms of fuel efficiency but also provides new insights into energy management for hybrid electric vehicles.

PDF

References

  1. S.Zhou, Z.Chen, D.Huang, and T.Lin, Model prediction and rule based energy management strategy for a plug-in hybrid electric vehicle with hybrid energy storage system, IEEE Trans. Power Electron., vol.36, no.5.pp. 5926–5940, May 2021.
  2. H. Tian, X. Wang, Z. Lu, Y. Huang, and G. Tian, “Adaptive fuzzy logic energy management strategy based on reasonable SOC reference curve for online control of plug - in hybrid electric city bus,” IEEE Trans. Intell. Transp. Syst., vol. 19, no. 5, pp. 1607 - 1617, May 2018.
  3. J. Liu, Y. Chen, J. Zhan, and F. Shang, “Heuristic dynamic programming based online energy management strategy for plug - in hybrid electric vehicles,” IEEE Trans. Veh. Technol., vol. 68, no. 5, pp. 4479 - 4493, May 2019.
  4. C. Hou, M. Ouyang, L. Xu, and H. Wang, “Approximate Pontryagin’s minimum principle applied to the energy management of plug - in hybrid electric vehicles,” Appl. Energy, vol. 115, pp. 174 - 189, 2014.
  5. Z. Chen, C. C. Mi, R. Xiong, J. Xu, and C. You, “Energy management of a power - split plug - in hybrid electric vehicle based on genetic algorithm and quadratic programming,” J. Power Sources, vol. 248, pp. 416 - 426, Feb. 2014.
  6. Li Y, Tao J, Xie L, Zhang R, Ma L, Qiao Z. Enhanced Q-learning for real-time hybrid electric vehicle energy management with deterministic rule[J]. Measurement and Control. 2020, 53(7-8):1493-1503.
  7. A. M. Phillips, M. Jankovic and K. E. Bailey, Vehicle system controller design for a hybrid electric vehicle[C]. IEEE International Conference on Control Applications. Conference Proceedings , Anchorage, AK, USA, 2000, pp. 297-302.
  8. Lv Ming, Yang Ying, Lijie Liang, et al. Energy Management Strategy of a Plug-in Parallel Hybrid Electric Vehicle Using Fuzzy Control[J]. Energy Procedia, 2017, 105.
  9. Gao C, Zhao J, Wu J. Optimal fuzzy logic based energy management strategy of battery/supercapacitor hybrid energy storage system for electric Vehicles[C].2016 12th World Congress on Intelligent Control and Automation, Guilin, China, June, 2016.
  10. Tang, X., Chu, L., Xu, N.et al. Energy Management of Planetary Gear Hybrid Electric Vehicle Based on Improved Dynamic Programming[C]. Neural Information Processing. ICONIP 2017.
  11. Shi, D., Guo, J., Liu, K., et al. Research on an Improved Rule-Based Energy Management Strategy Enlightened by the DP Optimization Results[J]. Sustainability 2023, 15.
  12. Y. Farajpour, H. Chaoui, M. Khayamy, S. Kelouwani and M. Alzayed, A Hybrid Energy Management Strategy Based on ANN and GA Optimization for Electric Vehicles[J]. 2022 IEEE Vehicle Power and Propulsion Conference (VPPC), Merced, CA, USA, 2022, pp. 1-6.
  13. Yapeng Li, Xiaolin Tang, Xianke Lin. The role and application of convex modeling and optimization in electrified vehicles[J]. Renewable and Sustainable Energy Reviews, Volume 153, 2022.
  14. Zhang Y, Chu L, Fu Z, et al. An improved adaptive equivalent consumption minimization strategy for parallel plug-in hybrid electric vehicle[C]. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering. 2019, 233(6): 1649 -1663.
  15. Zhang F, Hu X, Langari R, et al. Energy management strategies of connected HEVs and PHEVs: recent progress and outlook[J]. Prog Energy Combust Sci, 2019, 73:235-56.
  16. Zhou et al. Multi-step reinforcement learning for model-free predictive energy management of an electrified off-highway vehicle[J]. Applied Energy, 2019，25.
  17. Liu T, Zou Y, Liu D, et al. Reinforcement learning of adaptive energy management with transition probability for a hybrid electric tracked vehicle[J]. IEEE Trans Ind Electron, 2015, 62(12):7837-46.
  18. Xiong R, Cao J, Yu Q. Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle[J]. Applied Energy, 2018, 211:538-48.
  19. Wu J, He H, Peng J, et al. Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus[J]. Applied Energy, 2018, 222: 799-811.
  20. Da Wang, Lei Mei, Chuanxue Song, et al. Energy management strategy with mutation protection for fuel cell electric vehicles[J]. International Journal of Hydrogen Energy, 2024, 63:48-58.
  21. Tan H, Zhang H, Peng J, et al. Energy management of hybrid electric bus based on deep reinforcement learning in continuous state and action space[J]. Energy Convers Manag, 2019;195:548-60.
  22. Wu Y, Tan H, Peng J, et al. Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series parallel plug-in hybrid electric bus[J]. Applied Energy 2019, 247:454-66.
  23. Sun, X., et al. "A Markov chain-based velocity prediction model for hybrid electric vehicle energy management." Journal of Automotive Engineering, 2018.
  24. Zhang, H., et al. "ARIMA-based velocity prediction and its application in hybrid electric vehicle energy management." Energy Conversion and Management, 2020.
  25. Liu, Y., et al. "Support vector machine-based velocity prediction for hybrid electric vehicles." Transportation Research Part D, 2019.
  26. Chen, Z., et al. "Real-time velocity prediction using LSTM for hybrid electric vehicle control." IEEE Transactions on Intelligent Vehicles, 2021.
  27. Wang, Y., et al. "Vehicle-to-infrastructure velocity prediction for hybrid electric vehicle energy management." Transportation Research Part C, 2017.
  28. Ding, J., et al. "Random forest-based velocity prediction using connected vehicle data." Applied Energy, 2019.
  29. Wang, S., et al. "Integrated LSTM prediction and DRL optimization for hybrid electric vehicle energy management." IEEE Transactions on Vehicular Technology, 2022.
  30. Zou, X., et al. "Fuzzy logic-based predictive energy management for hybrid vehicles." Expert Systems with Applications, 2020.
  31. Ebbesen S, Elbert P, Guzzella L. Battery state-of-health perceptive energy management for hybrid electric vehicles[J]. IEEE Transactions on Vehicular technology, 2012, 61(7): 2893-2900.
  32. Z. Luo, H. Yan, and X. Pan, ‘Optimizing Transformer Models for Resource-Constrained Environments: A Study on Model Compression Techniques’, Journal of Computational Methods in Engineering Applications, pp. 1–12, Nov. 2023, doi: 10.62836/jcmea.v3i1.030107.
  33. H. Yan and D. Shao, ‘Enhancing Transformer Training Efficiency with Dynamic Dropout’, Nov. 05, 2024, arXiv: arXiv:2411.03236. doi: 10.48550/arXiv.2411.03236.
  34. Y. Liu and J. Wang, ‘AI-Driven Health Advice: Evaluating the Potential of Large Language Models as Health Assistants’, Journal of Computational Methods in Engineering Applications, pp. 1–7, Nov. 2023, doi: 10.62836/jcmea.v3i1.030106.
  35. Y. Gan and D. Zhu, ‘The Research on Intelligent News Advertisement Recommendation Algorithm Based on Prompt Learning in End-to-End Large Language Model Architecture’, Innovations in Applied Engineering and Technology, pp. 1–19, 2024.
  36. D. Zhu, Y. Gan, and X. Chen, ‘Domain Adaptation-Based Machine Learning Framework for Customer Churn Prediction Across Varing Distributions’, Journal of Computational Methods in Engineering Applications, pp. 1–14, 2021.
  37. H. Zhang, D. Zhu, Y. Gan, and S. Xiong, ‘End-to-End Learning-Based Study on the Mamba-ECANet Model for Data Security Intrusion Detection’, Journal of Information, Technology and Policy, pp. 1–17, 2024.
  38. D. Zhu, X. Chen, and Y. Gan, ‘A Multi-Model Output Fusion Strategy Based on Various Machine Learning Techniques for Product Price Prediction’, Journal of Electronic & Information Systems, vol. 4, no. 1.
  39. P. Ren and Z. Zhao, ‘Parental Recognition of Double Reduction Policy, Family Economic Status And Educational Anxiety: Exploring the Mediating Influence of Educational Technology Substitutive Resource’, Economics & Management Information, pp. 1–12, 2024.
  40. Z. Zhao, P. Ren, and Q. Yang, ‘Student self-management, academic achievement: Exploring the mediating role of self-efficacy and the moderating influence of gender insights from a survey conducted in 3 universities in America’, Apr. 17, 2024, arXiv: arXiv:2404.11029. doi: 10.48550/arXiv.2404.11029.
  41. P. Ren, Z. Zhao, and Q. Yang, ‘Exploring the Path of Transformation and Development for Study Abroad Consultancy Firms in China’, Apr. 17, 2024, arXiv: arXiv:2404.11034. doi: 10.48550/arXiv.2404.11034.
  42. Z. Zhao, P. Ren, and M. Tang, ‘How Social Media as a Digital Marketing Strategy Influences Chinese Students’ Decision to Study Abroad in the United States: A Model Analysis Approach’, Journal of Linguistics and Education Research, vol. 6, no. 1, pp. 12–23, 2024.
  43. Z. Zhao, P. Ren, and M. Tang, ‘Analyzing the Impact of Anti-Globalization on the Evolution of Higher Education Internationalization in China’, Journal of Linguistics and Education Research, vol. 5, no. 2, pp. 15–31, 2022.
  44. M. Tang, P. Ren, and Z. Zhao, ‘Bridging the gap: The role of educational technology in promoting educational equity’, The Educational Review, USA, vol. 8, no. 8, pp. 1077–1086, 2024.

Most read articles by the same author(s)