Advancing additive manufacturing: a Markov decision process approach for real-time quality assurance
Abstract
The study proposes a Markov Decision Process framework-based method for ensuring AM features are accurate in real time. The capability to make decisions in stochastic circumstances is facilitated by the Markov Decision Process framework, a mathematical model. Recasting the issue of AM quality assurance as a Markov decision process enables the objective of real-time optimization of process parameters and material attributes to ensure high-quality printing. In adjusting printing process parameters and material properties, the Markov Decision Process model considers the state of space, action space, transition probabilities, and rewards associated with such modifications. The objective is to identify the optimal policy that maximizes quality output while minimizing errors and rejections. In particular, the proposed approach accomplishes this through the utilization of machine learning and sensor data analysis. Temperature and pressure are among the process parameters and material qualities that are monitored in real time by sensors integrated into the additive manufacturing system. To ascertain any inconsistencies in quality among the data gathered by the sensors, statistical methodologies and machine learning algorithms are implemented. Regression analysis and control charts are two examples. Optimization is an additional component of the strategy. The Markov Decision Process framework incorporates optimization algorithms, such as value iteration, to determine the optimal approach for decision-making. By continuously updating the value function and policy, the technique learns and adapts to the dynamic nature of the printing process, ensuring a steady improvement in quality.
Downloads
References
Kumar, S., Gopi, T., Harikeerthana, N., et al. Machine learning techniques in additive manufacturing: a state-of-the-art review on design, processes and production control. J. Intell. Manuf., 34, 21–55 (2023).
Yao, B., Imani, F., & Yang, H. Markov decision process for image-guided additive manufacturing. IEEE Robotics and Automation Letters, 3(4), 2792–2798 (2018).
Framinan, J. M., Perez-Gonzalez, P., & Fernandez-Viagas, V. An overview on the use of operations research in additive manufacturing. Ann. Oper. Res., 322, 5–40 (2023).
Leung, Y. S., Kwok, T. H., Li, X., Yang, Y., Wang, C. C., & Chen, Y. Challenges and status on design and computation for emerging additive manufacturing technologies. J. Comput. Inf. Sci. Eng., 19(2), 021013 (2019).
Yan, H., Yang, J., & Wan, J. KnowIME: a system to construct a knowledge graph for intelligent manufacturing equipment. IEEE Access, 8, 41805–41813 (2020).
Barto, A. G. Reinforcement learning: An introduction. by Richard Sutton. SIAM Review, 6(2), 423 (2021).
Sutton, R. S., & Barto, A. G. Reinforcement Learning: An Introduction. MIT Press (2018).
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. Planning and acting in partially observable stochastic domains. Artif. Intell., 101(1–2), 99–134 (1998).
Luo, L., et al. Markov decision process for medical decision-making: A review. Artif. Intell. Med., 105, 101834 (2020).
Wu, L., et al. Markov decision processes in grid-interactive buildings and power systems: A review. Appl. Energy, 195, 1054–1067 (2017).
Chen, B., et al. Traffic signal control: A review of the state of the art. Transp. Res. Part C Emerg. Technol., 103, 187–205 (2019).
Gibson, I., Rosen, D., & Stucker, B. Direct digital manufacturing. In: Additive Manufacturing Technologies: 3D Printing, Rapid Prototyping, and Direct Digital Manufacturing, pp. 375–397. Springer (2015).
Qin, Y., Qi, Q., Scott, P. J., & Jiang, X. Determination of optimal build orientation for additive manufacturing using Muirhead mean and prioritised average operators. J. Intell. Manuf., 30, 3015–3034 (2019).
Gu, D. Additive Manufacturing: Materials, Processes, Quantifications and Applications. PLoS One, 13(8), e0201587 (2018).
Zhang, S., & Sun, C. Recent advances in multi-material and multi-scale 3D printing. Virtual and Physical Prototyping, (2017).
Tumbleston, J. R., Shirvanyants, D., Ermoshkin, N., Janusziewicz, R., Johnson, A. R., Kelly, D., et al. Continuous liquid interface production of 3D objects. Science, 347(6228), 1349–1352 (2015).
Campbell, I., Bourell, D., & Gibson, I. Additive manufacturing: rapid prototyping comes of age. Rapid Prototyping Journal, 18(4), 255–258 (2012).
Maleki Pour, E. Innovative tessellation algorithm for generating more uniform temperature distribution in the powderbed fusion process. Doctoral dissertation (2018).
Mellor, S., Hao, L., & Zhang, D. Additive manufacturing: A framework for implementation. Int. J. Prod. Econ., 149, 194–201 (2014).
Thompson, M. K., Moroni, G., Vaneker, T., Fadel, G., Campbell, R. I., Gibson, I., & Martina, F. Design for Additive Manufacturing: Trends, opportunities, considerations, and constraints. CIRP Annals, 65(2), 737–760 (2016).
Koumoulos, P., Bandyopadhyay, A., Bose, S., O’Donoghue, L., & Charitidis, C. Additive manufacturing: scientific and technological challenges, market uptake and opportunities. Addit. Manuf., 21(1), 22–23 (2021).
Lee, J., & Ni, J. Real-time quality assurance for additive manufacturing processes. Int. J. Prod., 74(4) (2018).
Chen, L., et al. Real-time quality assurance for additive manufacturing based on machine learning and big data analytics. J. Manuf. Syst., 62, 728–752 (2020).
Xu, B., et al. Real-time quality assurance for additive manufacturing via data fusion and adaptive control. Int. J. Prod., 57, 31–35 (2020).
Kumar, S., et al. A review of quality assurance in additive manufacturing. Rapid Prototyping, 17–35 (2019).
Rao, P. K., Liu, J., Roberson, D., Kong, Z., & Williams, C. Online real-time quality monitoring in additive manufacturing processes using heterogeneous sensors. J. Manuf. Sci. Eng., 137(6), 061007 (2015).
Lee, J., Noh, S. D., Kim, H. J., & Kang, Y. S. Implementation of cyber-physical production systems for quality prediction and operation control in metal casting. Sensors, 18(5), 1428 (2018).
Li, X., Jia, X., Yang, Q., & Lee, J. Quality analysis in metal additive manufacturing with deep learning. J. Intell. Manuf., 31(8), 2003–2017 (2020).
Liu, J., Xu, Y., Ge, Y., Hou, Z., & Chen, S. Wire and arc additive manufacturing of metal components: a review of recent research developments. Int. J. Adv. Manuf. Technol., 111(1), 149–198 (2020).
Chourasia, A., Kumar, P., & Cad, A. (2019). Analysis and review of rapid prototyping technology & study of materia used in process of 3D printing. International Research Journal of Engineering, 6(10), 573–579.
Rao, P. K., Liu, J., Roberson, D., Kong, Z., & Williams, C. (2015). Online real-time quality monitoring in additive manufacturing processes using heterogeneous sensors. Journal of Manufacturing Science and Engineering, 137(6), 061007.
Li, Y., Zhang, F. P., Yan, Y., Zhou, J. H., & Li, Y. F. (2019). Multi-source uncertainty considered assembly process quality control based on surrogate model and information entropy. Structural and Multidisciplinary Optimization, 59, 1685–1701.
Das, S., et al. (2018). Optimal control strategies for surface quality in additive manufacturing. Journal of Manufacturing Science and Engineering, 140(12), 121001.
Kumar, A., et al. (2019). Markov decision process-based framework for quality control in additive manufacturing. Journal of Manufacturing Science and Engineering, 141(12), 121377.
Wang, C., Tan, X. P., Tor, S. B., & Lim, C. S. (2020). Machine learning in additive manufacturing: State-of-the-art and perspectives. Additive Manufacturing, 36, 101538.
Das, S., et al. (2018). Optimal control strategies for surface quality in additive manufacturing. Journal of Manufacturing Science and Engineering, 140(12), 121001.
Zhang, X., et al. (2020). A Markov decision process-based framework for quality control in additive manufacturing with online monitoring. Journal of Manufacturing Science and Engineering, 142(6), 061005.
Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine Learning, 8, 279–292.
Rummery, G. A., & Niranjan, M. (1994). Online Q-learning using connectionist systems. Cambridge, England: University of Cambridge, Department of Engineering.
Mnih, V., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., ... & Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533.
Cully, A., Clune, J., Tarapore, D., & Mouret, J. B. (2015). Robots that can adapt like animals. Nature, 521(7553), 503–507.
Deisenroth, M. P., Neumann, G., & Peters, J. (2013). A survey on policy search for robotics. Foundations and Trends in Robotics, 2(1–2), 1–142.
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., ... & Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971.
Schulman, J., et al. (2015). Trust region policy optimization. Proceedings of the 32nd International Conference on Machine Learning, pp. 1889–1897.
Heess, N., et al. (2017). Emergence of locomotion behaviours in rich environments. arXiv:1707.02286.
Altintas, Y. (2012). Manufacturing automation: Metal cutting mechanics, machine tool vibrations, and CNC design. Cambridge University Press.
A. (2019). Smart manufacturing. International Journal of Production Research, pp. 529–533.
Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 8(3–4), 279–292.
Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction. MIT Press.
Lillicrap, T. P., et al. (2015). Continuous control with deep reinforcement learning. arXiv:1509.02971.
Liu, C., et al. (2022). Digital twin-enabled collaborative data management for metal additive manufacturing systems. Journal of Manufacturing Systems, 62, 857–874.
Kusiak, A. (2019). Fundamentals of smart manufacturing: A multi-thread perspective. Annual Reviews in Control, 47, 214–220.
Lu, Y., et al. (2020). Digital Twin-driven smart manufacturing: Connotation, reference model, applications and research issues. Robotics and Computer-Integrated Manufacturing, 61, 101837.
Huang, Z., et al. (2021). A survey on AI-driven digital twins in industry 4.0: Smart manufacturing and advanced robotics. Sensors, 21(19), 6340.
Kusiak, A. (2020). Convolutional and generative adversarial neural networks in manufacturing. International Journal of Production Research, 58(5), 1594–1604.
Guo, D., Zhang, W., & Yang, S. (2018). A review on topology optimization for additive manufacturing. Structural and Multidisciplinary Optimization.
Zhang, W., Li, Z., & Guo, D. (2019). A review of process parameter optimization techniques in additive manufacturing. Additive Manufacturing, 27.
Li, R., Tang, Y., & Chen, Y. (2020). Support structure optimization in additive manufacturing: a review. Journal of Manufacturing Systems, 54.
Tanskanen, P., Karki, T., & Partanen, J. (2018). Review of build orientation optimization in additive manufacturing. Additive Manufacturing, 34(6), 1–23.
Trevisan, F., et al. (2017). On the selective laser melting (SLM) of the AlSi10Mg alloy: process, microstructure, and mechanical properties. Materials, 10(1), 76.
Borle, S., Patil, S., & Gao, R. X. (2020). A review of machine learning applications in additive manufacturing using big data. Additive Manufacturing, 21(19), 7231.
Tapia, G., & Elwany, A. (2019). Simulation-based optimization for additive manufacturing: A review. Journal of Manufacturing Systems, 12(23), 1297.
Huang, Y., et al. (2015). Additive manufacturing: current state, future potential, gaps and needs, and recommendations. Journal of Manufacturing Science and Engineering, 137(1), 014001.
Ahuja, B., & Anand, S. (2017). Design for additive manufacturing: A review of design strategies and their implications on product development. International Journal of Rapid Manufacturing, 42, 1–25.
Taminger, K. M., Das, S., & Babu, S. S. (2017). Modeling and simulation of metal additive manufacturing and powder bed fusion processes. Annual Review of Materials Research, 12(28), 1728.
Gor, M., et al. (2022). Density prediction in powder bed fusion additive manufacturing: machine learning-based techniques. Applied Sciences, 12(14), 7271.
Copyright (c) 2025 Boletim da Sociedade Paranaense de Matemática
This work is licensed under a Creative Commons Attribution 4.0 International License.
When the manuscript is accepted for publication, the authors agree automatically to transfer the copyright to the (SPM).
The journal utilize the Creative Common Attribution (CC-BY 4.0).
