Vol. 5 No. 1 (2025): Issue 5
Articles

Hidden Markov Model-based approach for Efficient f Lexical Analysis

PDF
  • Wei Li,
  • Min Zhang,
  • Fang Wang
Wei Li
Institute of Computational Linguistics, Shaoyang University
Min Zhang
Center for Artificial Intelligence Research, Hengshui College
Fang Wang
Advanced Language Processing Lab, Yulin Normal University

Published 2025-03-08

Keywords

  • Lexical Analysis,
  • Natural Language Processing,
  • Information Extraction,
  • Hidden Markov Model,
  • Machine Learning Techniques

How to Cite

Li, W., Zhang, M., & Wang, F. (2025). Hidden Markov Model-based approach for Efficient f Lexical Analysis. Generative Artificial Intelligence Research, 5(1). Retrieved from https://ojs.mri-pub.com/index.php/GAIR/article/view/105

Abstract

Efficient lexical analysis plays a crucial role in various natural language processing applications. However, the existing research has encountered challenges in accurately identifying and extracting the meaningful information from vast amounts of textual data. This paper addresses the need for a more effective approach by proposing a Hidden Markov Model-based method for lexical analysis. The innovation lies in leveraging the power of probabilistic graphical models to capture the complex relationships among words and improve the accuracy of information extraction. Our work focuses on developing a novel algorithm that combines Hidden Markov Models with advanced machine learning techniques to enhance the efficiency and accuracy of lexical analysis tasks. This research contributes to advancing the field of natural language processing and opens up new avenues for improving the performance of text analysis systems.

PDF

References

  1. R. Ferreira et al., "Assessing sentence similarity through lexical, syntactic and semantic analysis," Comput. Speech Lang., vol. 39, pp. 1-28, 2016.
  2. V. Bambini et al., "Deconstructing heterogeneity in schizophrenia through language: a semi-automated linguistic analysis and data-driven clustering approach," Schizophrenia, vol. 8, 2022.
  3. M. R. Ashraf et al., "BERT-Based Sentiment Analysis for Low-Resourced Languages: A Case Study of Urdu Language," IEEE Access, vol. 11, pp. 110245-110259, 2023.
  4. A. McMahon et al., "Gestural representation and Lexical Phonology," Phonology, vol. 11, pp. 277-316, 1994.
  5. Y. Masoud and F. Mostafa, "Examining the Effect of Ideology and Idiosyncrasy on Lexical Choices in Translation Studies within the CDA Framework," The Journal of English Studies, vol. 1, pp. 27-36, 2011.
  6. J. Wang et al., "Reading Chinese Script : A Cognitive Analysis," 1999.
  7. L. Miilher and F. Fernandes, "Pragmatic, lexical and grammatical abilities of autistic spectrum children," Pro-fono : revista de atualizacao cientifica, vol. 21, no. 4, pp. 309-314, 2009.
  8. R. Weinstein et al., "The Interaction of Vocal Context and Lexical Predictability," Language and Speech, vol. 8, pp. 56-67, 1965.
  9. A. Krogh et al., "Predicting transmembrane protein topology with a hidden Markov model: application to complete genomes," J. Mol. Biol., vol. 305, no. 3, 2001.
  10. Y. Zhang et al., "Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm," IEEE Trans. Med. Imaging, vol. 20, 2001.
  11. E. Sonnhammer et al., "A Hidden Markov Model for Predicting Transmembrane Helices in Protein Sequences," Proc. Int. Conf. Intell. Syst. Mol. Biol., vol. 6, 1998.
  12. K. Wang et al., "PennCNV: an integrated hidden Markov model designed for high-resolution copy number variation detection in whole-genome SNP genotyping data," Genome Res., vol. 17, no. 11, 2007.
  13. J. Cheng et al., "Hidden Markov Model-Based Nonfragile State Estimation of Switched Neural Network With Probabilistic Quantized Outputs," IEEE Trans. Cybern., vol. 50, 2020.
  14. V. M. Narasimhan et al., "BCFtools/RoH: a hidden Markov model approach for detecting autozygosity from next-generation sequencing data," Bioinformatics, vol. 32, 2016.
  15. C. Yang and G. Gidófalvi, "Fast map matching, an algorithm integrating hidden Markov model with precomputation," Int. J. Geogr. Inf. Sci., vol. 32, 2018.
  16. S. Dong et al., "Quantized Control of Markov Jump Nonlinear Systems Based on Fuzzy Hidden Markov Model," IEEE Trans. Cybern., vol. 49, 2019.
  17. Z. Luo, H. Yan, and X. Pan, ‘Optimizing Transformer Models for Resource-Constrained Environments: A Study on Model Compression Techniques’, Journal of Computational Methods in Engineering Applications, pp. 1–12, Nov. 2023, doi: 10.62836/jcmea.v3i1.030107.
  18. H. Yan and D. Shao, ‘Enhancing Transformer Training Efficiency with Dynamic Dropout’, Nov. 05, 2024, arXiv: arXiv:2411.03236. doi: 10.48550/arXiv.2411.03236.
  19. H. Yan, ‘Real-Time 3D Model Reconstruction through Energy-Efficient Edge Computing’, Optimizations in Applied Machine Learning, vol. 2, no. 1, 2022.
  20. W. Cui, J. Zhang, Z. Li, H. Sun, and D. Lopez, ‘Kamalika Das, Bradley Malin, and Sricharan Kumar. 2024. Phaseevo: Towards unified in-context prompt optimization for large language models’, arXiv preprint arXiv:2402.11347.
  21. Z. Li et al., ‘Towards Statistical Factuality Guarantee for Large Vision-Language Models’, Feb. 27, 2025, arXiv: arXiv:2502.20560. doi: 10.48550/arXiv.2502.20560.
  22. W. Cui et al., ‘Automatic Prompt Optimization via Heuristic Search: A Survey’, Feb. 26, 2025, arXiv: arXiv:2502.18746. doi: 10.48550/arXiv.2502.18746.
  23. Y.-S. Cheng, P.-M. Lu, C.-Y. Huang, and J.-J. Wu, ‘Encapsulation of lycopene with lecithin and α-tocopherol by supercritical antisolvent process for stability enhancement’, The Journal of Supercritical Fluids, vol. 130, pp. 246–252, 2017.
  24. P.-M. Lu and Z. Zhang, ‘The Model of Food Nutrition Feature Modeling and Personalized Diet Recommendation Based on the Integration of Neural Networks and K-Means Clustering’, Journal of Computational Biology and Medicine, vol. 5, no. 1, 2025.
  25. P.-M. Lu, ‘Potential Benefits of Specific Nutrients in the Management of Depression and Anxiety Disorders’, Advanced Medical Research, vol. 3, no. 1, pp. 1–10, 2024.
  26. P.-M. Lu, ‘Exploration of the Health Benefits of Probiotics Under High-Sugar and High-Fat Diets’, Advanced Medical Research, vol. 2, no. 1, pp. 1–9, 2023.
  27. P.-M. Lu, ‘The Preventive and Interventional Mechanisms of Omega-3 Polyunsaturated Fatty Acids in Krill Oil for Metabolic Diseases’, Journal of Computational Biology and Medicine, vol. 4, no. 1, 2024.
  28. C. Li and Y. Tang, ‘The Factors of Brand Reputation in Chinese Luxury Fashion Brands’, Journal of Integrated Social Sciences and Humanities, pp. 1–14, 2023.
  29. Y. Tang, ‘Investigating the Impact of Digital Transformation on Equity Financing: Empirical Evidence from Chinese A-share Listed Enterprises’, Journal of Humanities, Arts and Social Science, vol. 8, no. 7, pp. 1620–1632, 2024.
  30. Y. Tang and C. Li, ‘Exploring the Factors of Supply Chain Concentration in Chinese A-Share Listed Enterprises’, Journal of Computational Methods in Engineering Applications, pp. 1–17, 2023.
  31. C. Li and Y. Tang, ‘Emotional Value in Experiential Marketing: Driving Factors for Sales Growth–A Quantitative Study from the Eastern Coastal Region’, Economics & Management Information, pp. 1–13, 2024.
  32. Y. C. Li and Y. Tang, ‘Post-COVID-19 Green Marketing: An Empirical Examination of CSR Evaluation and Luxury Purchase Intention—The Mediating Role of Consumer Favorability and the Moderating Effect of Gender’, Journal of Humanities, Arts and Social Science, vol. 8, no. 10, pp. 2410–2422, 2024.
  33. C. Li, Y. Tang, and K. Xu, ‘Investigating the impact AI on Corporate financial and operating flexibility of Retail Enterprises in China’, Economic and Financial Research Letters, vol. 5, no. 1, 2025.
  34. Y. Tang and K. Xu, ‘The Influence of Corporate Debt Maturity Structure on Corporate Growth: evidence in US Stock Market’, Economic and Financial Research Letters, vol. 1, no. 1, 2024.
  35. C. Kim, Z. Zhu, W. B. Barbazuk, R. L. Bacher, and C. D. Vulpe, ‘Time-course characterization of whole-transcriptome dynamics of HepG2/C3A spheroids and its toxicological implications’, Toxicology Letters, vol. 401, pp. 125–138, 2024.
  36. J. Shen et al., ‘Joint modeling of human cortical structure: Genetic correlation network and composite-trait genetic correlation’, NeuroImage, vol. 297, p. 120739, 2024.
  37. K. F. Faridi et al., ‘Factors associated with reporting left ventricular ejection fraction with 3D echocardiography in real‐world practice’, Echocardiography, vol. 41, no. 2, p. e15774, Feb. 2024, doi: 10.1111/echo.15774.
  38. Y. Gan and D. Zhu, ‘The Research on Intelligent News Advertisement Recommendation Algorithm Based on Prompt Learning in End-to-End Large Language Model Architecture’, Innovations in Applied Engineering and Technology, pp. 1–19, 2024.
  39. H. Zhang, D. Zhu, Y. Gan, and S. Xiong, ‘End-to-End Learning-Based Study on the Mamba-ECANet Model for Data Security Intrusion Detection’, Journal of Information, Technology and Policy, pp. 1–17, 2024.
  40. D. Zhu, Y. Gan, and X. Chen, ‘Domain Adaptation-Based Machine Learning Framework for Customer Churn Prediction Across Varing Distributions’, Journal of Computational Methods in Engineering Applications, pp. 1–14, 2021.
  41. D. Zhu, X. Chen, and Y. Gan, ‘A Multi-Model Output Fusion Strategy Based on Various Machine Learning Techniques for Product Price Prediction’, Journal of Electronic & Information Systems, vol. 4, no. 1.
  42. X. Chen, Y. Gan, and S. Xiong, ‘Optimization of Mobile Robot Delivery System Based on Deep Learning’, Journal of Computer Science Research, vol. 6, no. 4, pp. 51–65, 2024.
  43. Y. Gan, J. Ma, and K. Xu, ‘Enhanced E-Commerce Sales Forecasting Using EEMD-Integrated LSTM Deep Learning Model’, Journal of Computational Methods in Engineering Applications, pp. 1–11, 2023.
  44. F. Zhang et al., ‘Natural mutations change the affinity of μ-theraphotoxin-Hhn2a to voltage-gated sodium channels’, Toxicon, vol. 93, pp. 24–30, 2015.
  45. Y. Gan and X. Chen, ‘The Research on End-to-end Stock Recommendation Algorithm Based on Time-frequency Consistency’, Advances in Computer and Communication, vol. 5, no. 4, 2024.
  46. Z. Zhao, P. Ren, and Q. Yang, ‘Student self-management, academic achievement: Exploring the mediating role of self-efficacy and the moderating influence of gender insights from a survey conducted in 3 universities in America’, Apr. 17, 2024, arXiv: arXiv:2404.11029. doi: 10.48550/arXiv.2404.11029.
  47. Z. Zhao, P. Ren, and M. Tang, ‘Analyzing the Impact of Anti-Globalization on the Evolution of Higher Education Internationalization in China’, Journal of Linguistics and Education Research, vol. 5, no. 2, pp. 15–31, 2022.
  48. M. Tang, P. Ren, and Z. Zhao, ‘Bridging the gap: The role of educational technology in promoting educational equity’, The Educational Review, USA, vol. 8, no. 8, pp. 1077–1086, 2024.
  49. P. Ren, Z. Zhao, and Q. Yang, ‘Exploring the Path of Transformation and Development for Study Abroad Consultancy Firms in China’, Apr. 17, 2024, arXiv: arXiv:2404.11034. doi: 10.48550/arXiv.2404.11034.
  50. P. Ren and Z. Zhao, ‘Parental Recognition of Double Reduction Policy, Family Economic Status And Educational Anxiety: Exploring the Mediating Influence of Educational Technology Substitutive Resource’, Economics & Management Information, pp. 1–12, 2024.
  51. Z. Zhao, P. Ren, and M. Tang, ‘How Social Media as a Digital Marketing Strategy Influences Chinese Students’ Decision to Study Abroad in the United States: A Model Analysis Approach’, Journal of Linguistics and Education Research, vol. 6, no. 1, pp. 12–23, 2024.
  52. J. Lei, ‘Efficient Strategies on Supply Chain Network Optimization for Industrial Carbon Emission Reduction’, JCMEA, pp. 1–11, Dec. 2022.
  53. J. Lei, ‘Green Supply Chain Management Optimization Based on Chemical Industrial Clusters’, IAET, pp. 1–17, Nov. 2022, doi: 10.62836/iaet.v1i1.003.
  54. J. Lei and A. Nisar, ‘Investigating the Influence of Green Technology Innovations on Energy Consumption and Corporate Value: Empirical Evidence from Chemical Industries of China’, Innovations in Applied Engineering and Technology, pp. 1–16, 2023.
  55. J. Lei and A. Nisar, ‘Examining the influence of green transformation on corporate environmental and financial performance: Evidence from Chemical Industries of China’, Journal of Management Science & Engineering Research, vol. 7, no. 2, pp. 17–32, 2024.
  56. Y. Jia and J. Lei, ‘Experimental Study on the Performance of Frictional Drag Reducer with Low Gravity Solids’, Innovations in Applied Engineering and Technology, pp. 1–22, 2024.