Energy Consumption Optimization for Flexible Job-Shop Scheduling in Manufacturing Industry Using Multi-Agent Reinforcement Learning
DOI:
https://doi.org/10.62146/ijecbe.v4i1.227Keywords:
flexible job-shop scheduling problem (FJSSP), energy optimization, Multi-Agent Reinforcement Learning (MARL)Abstract
Modern manufacturing industries are increasingly demanding production systems that are both flexible and efficient in handling dynamic operational conditions. Traditional scheduling approaches, such as the job-shop scheduling Problem (JSSP), are limited by the constraint that each operation must be executed on a predefined machine. To address this limitation, the flexible job-shop scheduling Problem (FJSSP) was introduced, allowing alternative machine options for each operation and thereby enhancing system flexibility. This study proposes a scheduling optimization approach based on Multi-Agent Reinforcement Learning (MARL), to support decision-making in complex production environments. Experimental results demonstrate that the proposed method reduces energy consumption by up to 40.62% compared to the First Come First Serve (FCFS) method and by 35.23% compared to the Fastest Available Agent (FAA) method. Moreover, the model shows superior performance in controlling theworst-case makespan and achieves significantly higher success rates in satisfying various production constraints compared to all tested rule-based methods.
References
Fangfang Zhang et al. “Survey on Genetic Programming and Machine Learning Techniques for Heuristic Design in Job Shop Scheduling”. In: IEEE Transactions on Evolutionary Computation 28.1 (2024), pp. 147–167. DOI: 10.1109/TEVC.2023.3255246.
Yaping Fu et al. “Distributed scheduling problems in intelligent manufacturing systems”. In: Tsinghua Science and Technology 26.5 (2021), pp. 625–645. DOI: 10.26599/TST.2021.9010009.
Ling Wang, Zixiao Pan, and Jing jing Wang. “A Review of Reinforcement Learning Based Intelligent Optimization for Manufacturing Scheduling”. In: Complex System Modeling and Simulation 1.4 (2021),pp. 257–270. DOI: 10.23919/CSMS.2021.0027.
Zihui Luo et al. “Flow-Shop Scheduling Problem With Batch Processing Machines via Deep Reinforcement Learning for Industrial Internet of Things”. In: IEEE Transactions on Emerging Topics in Computational Intelligence 8.5 (2024), pp. 3518–3533. DOI: 10.1109/TETCI.2024.3402685.
Ruiqi Chen, Wenxin Li, and Hongbing Yang. “A Deep Reinforcement Learning Framework Based on an Attention Mechanism and Disjunctive Graph Embedding for the Job-Shop Scheduling Problem”. In: IEEE Transactions on Industrial Informatics 19.2 (2023), pp. 1322–1331. DOI: 10.1109/TII.2022.3167380.
Chien-Liang Liu, Chuan-Chin Chang, and Chun-Jan Tseng. “Actor-Critic Deep Reinforcement Learning for Solving Job Shop Scheduling Problems”. In: IEEE Access 8 (2020), pp. 71752–71762.DOI: 10.1109/ACCESS.2020.2987820.
Chien-Liang Liu, Chun-Jan Tseng, and Po-Hao Weng. “Dynamic Job-Shop Scheduling via Graph Attention Networks and Deep Reinforcement Learning”. In: IEEE Transactions on Industrial Informatics 20.6 (2024), pp. 8662–8672. DOI: 10.1109/TII.2024.3371489.
Kun Lei et al. “Large-Scale Dynamic Scheduling for Flexible Job-Shop With Random Arrivals of New Jobs by Hierarchical Reinforcement Learning”. In: IEEE Transactions on Industrial Informatics 20.1 (2024), pp. 1007–1018. DOI: 10.1109/TII.2023.3272661.
Yejian Zhao et al. “Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network”. In: IEEE Access 9 (2021), pp. 122995–123011. DOI: 10.1109/ACCESS.2021.3110242.
Kuo-Hao Ho et al. “Residual Scheduling: A New Reinforcement Learning Approach to Solving Job Shop Scheduling Problem”. In: IEEE Access 12 (2024), pp. 14703–14718. DOI: 10.1109/ACCESS.2024.3357969.
Stéphane Dauzère-Pérès et al. “The flexible job shop scheduling problem: A review”. In: European Journal of Operational Research 314.2 (2024), pp. 409–432. ISSN: 0377-2217. DOI: https://doi.org/10.1016/j.ejor.2023.05.017.
Kun Lei et al. “A multi-action deep reinforcement learning framework for flexible Job-shop scheduling problem”. In: Expert Systems with Applications 205 (2022), p. 117796. ISSN: 0957-4174. DOI: https://doi.org/10.1016/j.eswa.2022.117796.
Anbang Liu et al. “Integrating Machine Learning and Mathematical Optimization for Job Shop Scheduling”. In: IEEE Transactions on Automation Science and Engineering 21.3 (2024), pp. 4829–4850. DOI: 10.1109/TASE.2023.3303175.
ChengRan Lin, ZhengCai Cao, and MengChu Zhou. “Learning-Based Grey Wolf Optimizer for Stochastic Flexible Job Shop Scheduling”. In: IEEE Transactions on Automation Science and Engineering 19.4 (2022), pp. 3659–3671. DOI: 10.1109/TASE.2021.3129439.
Wen Song et al. “Flexible Job-Shop Scheduling via Graph Neural Network and Deep Reinforcement Learning”. In: IEEE Transactions on Industrial Informatics 19.2 (2023), pp. 1600–1610. DOI: 10.1109/TII.2022.3189725.
Junwen Ding et al. “A Novel Evolutionary Algorithm for Energy-Efficient Scheduling in Flexible Job Shops”. In: IEEE Transactions on Evolutionary Computation 27.5 (2023), pp. 1470–1484. DOI: 10.1109/TEVC.2022.3222791.
Liji Shen, Stéphane Dauzère-Pérès, and Söhnke Maecker. “Energy cost efficient scheduling in flexible job-shop manufacturing systems”. In: European Journal of Operational Research 310.3 (2023),pp. 992–1016. ISSN: 0377-2217. DOI: https://doi.org/10.1016/j.ejor.2023.03.041.
Jianhua Wang, Chuanyu Wu, and Yongtao Peng. “Multi-objective scheduling for an energy-efficient flexible job shop problem with peak power constraint”. In: Applied Soft Computing 167 (2024), p. 112330. ISSN: 1568-4946. DOI: https://doi.org/10.1016/j.asoc.2024.112330.
Moisés Santana Pereira et al. “Energy Estimation and Production Scheduling in Job Shop Using Machine Learning”. In: IEEE Access 12 (2024), pp. 104177–104189. DOI: 10.1109/ACCESS.2024.3430218.
Dazzle Johnson, Gang Chen, and Yuqian Lu. “Multi-Agent Reinforcement Learning for Real-Time Dynamic Production Scheduling in a Robot Assembly Cell”. In: IEEE Robotics and Automation Letters 7.3 (2022), pp. 7684–7691. DOI: 10.1109/LRA.2022.3184795.
Aaron Hao Tan et al. “Deep Reinforcement Learning for Decentralized Multi-Robot Exploration With Macro Actions”. In: IEEE Robotics and Automation Letters 8.1 (2023), pp. 272–279. DOI: 10.1109/LRA.2022.3224667.
Yuxin Li et al. “Real-Time Scheduling for Flexible Job Shop With AGVs Using Multiagent Reinforcement Learning and Efficient Action Decoding”. In: IEEE Transactions on Systems, Man, and Cybernetics: Systems 55.3 (2025), pp. 2120–2132. DOI: 10.1109/TSMC.2024.3520381.
Kok-Lim Alvin Yau et al. “The Augmented Intelligence Perspective on Human-in-the-Loop Reinforcement Learning: Review, Concept Designs, and Future Directions”. In: IEEE Transactions on Human-Machine Systems 54.6 (2024), pp. 762–777. DOI: 10.1109/THMS.2024.3467370.
Xu Wang et al. “Deep Reinforcement Learning: A Survey”. In: IEEE Transactions on Neural Networks and Learning Systems 35.4 (2024), pp. 5064–5078. DOI: 10.1109/TNNLS.2022.3207346.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 International Journal of Electrical, Computer, and Biomedical Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
