Adaptive reinforcement learning for task scheduling in aircraft maintenance.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
03 Oct 2023
03 Oct 2023
Historique:
received:
01
05
2023
accepted:
23
08
2023
medline:
4
10
2023
pubmed:
4
10
2023
entrez:
3
10
2023
Statut:
epublish
Résumé
This paper proposes using reinforcement learning (RL) to schedule maintenance tasks, which can significantly reduce direct operating costs for airlines. The approach consists of a static algorithm for long-term scheduling and an adaptive algorithm for rescheduling based on new maintenance information. To assess the performance of both approaches, three key performance indicators (KPIs) are defined: Ground Time, representing the hours an aircraft spends on the ground; Time Slack, measuring the proximity of tasks to their due dates; and Change Score, quantifying the similarity level between initial and adapted maintenance plans when new information surfaces. The results demonstrate the efficacy of RL in producing efficient maintenance plans, with the algorithms complementing each other to form a solid foundation for routine tasks and real-time responsiveness to new information. While the static algorithm performs slightly better in terms of Ground Time and Time Slack, the adaptive algorithm excels overwhelmingly in terms of Change Score, offering greater flexibility in handling new maintenance information. The proposed RL-based approach can improve the efficiency of aircraft maintenance and has the potential for further research in this area.
Identifiants
pubmed: 37789033
doi: 10.1038/s41598-023-41169-3
pii: 10.1038/s41598-023-41169-3
pmc: PMC10547829
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
16605Subventions
Organisme : European Union H2020
ID : 769288
Organisme : Fundação para a Ciência e a Tecnologia
ID : 00326
Informations de copyright
© 2023. Springer Nature Limited.
Références
IATA: Airline Maintenance Cost Executive Commentary. https://www.iata.org/contentassets/bf8ca67c8bcd4358b3d004b0d6d0916f/fy2020-mctg-report_public.pdf .
Sriram, C. & Haghani, A. An optimization model for aircraft maintenance scheduling and re-assignment. Transp. Res. Part A 37(1), 29–48. https://doi.org/10.1016/S0965-8564(02)00004-6 (2003).
doi: 10.1016/S0965-8564(02)00004-6
Safaei, N., Banjevic, D. & Jardine, A. Workforce-constrained maintenance scheduling for aircraft fleet: A case study. Ann. Oper. Res. 186(1), 295–316. https://doi.org/10.1007/s10479-011-0885-4 (2011).
doi: 10.1007/s10479-011-0885-4
Li, H., Zuo, H., Lei, D., Liang, K. & Lu, T. Optimal combination of aircraft maintenance tasks by a novel simplex optimization method. Math. Probl. Eng. 2015. https://doi.org/10.1155/2015/169310 (2015).
Witteman, M., Deng, Q. & Santos, B. F. A bin packing approach to solve the aircraft maintenance task allocation problem. Eur. J. Oper. Res. 294(1), 365–376. https://doi.org/10.1016/j.ejor.2021.01.027 (2021).
doi: 10.1016/j.ejor.2021.01.027
Knowles, M., Baglee, D. & Wermter, S. Reinforcement Learning for Scheduling of Maintenance. 409–422. https://doi.org/10.1007/978-0-85729-130-1_31 (2010).
Wang, X. & Qi, C. Multi-agent reinforcement learning based maintenance policy for a resource constrained flow line system. J. Intell. Manuf. 27(2), 325–333. https://doi.org/10.1007/s10845-013-0864-5 (2016).
doi: 10.1007/s10845-013-0864-5
Hu, Y., Miao, X., Zhang, J., Liu, J. & Pan, E. Reinforcement learning-driven maintenance strategy: A novel solution for long-term aircraft maintenance decision optimization. Comput. Ind. Eng. 153, 107056. https://doi.org/10.1016/j.cie.2020.107056 (2021).
doi: 10.1016/j.cie.2020.107056
Andrade, P., Silva, C., Ribeiro, B. & Santos, B. F. Aircraft maintenance check scheduling using reinforcement learning. Aerospace https://doi.org/10.3390/aerospace8040113 (2021).
doi: 10.3390/aerospace8040113
Lee, J. & Mitici, M. Deep reinforcement learning for predictive aircraft maintenance using probabilistic remaining-useful-life prognostics. Reliabil. Eng. Syst. Saf. 230, 108908. https://doi.org/10.1016/j.ress.2022.108908 (2023).
doi: 10.1016/j.ress.2022.108908
Zhang, Y., Li, C., Su, X., Cui, R. & Wan, B. A baseline-reactive scheduling method for carrier-based aircraft maintenance tasks. Complex Intell. Syst. 9, 367–397. https://doi.org/10.1007/s40747-022-00784-9 (2023).
doi: 10.1007/s40747-022-00784-9
Xue, B., Qiu, H., Niu, B. & Yan, X. Improved aircraft maintenance technician scheduling with task splitting strategy based on particle swarm optimization. In Advances in Swarm Intelligence. ICSI 2022. Lecture Notes in Computer Science. Vol. 13344. https://doi.org/10.1007/978-3-031-09677-8_18 (Springer, 2022).
Shahmoradi-Moghadam, H., Safaei, N. & Sadjadi, S. J. Robust maintenance scheduling of aircraft fleet: A hybrid simulation-optimization approach. IEEE Access 9, 17854–17865. https://doi.org/10.1109/ACCESS.2021.3053714 (2021).
doi: 10.1109/ACCESS.2021.3053714
Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 2018).
Watkins, C. & Dayan, P. Q-learning. Mach. Learn. 8, 279–292. https://doi.org/10.1007/BF00992698 (1992).
doi: 10.1007/BF00992698
Van Hasselt, H. Double Q-learning. In Proceedings of the 4th Annual Conference Advance Neural Information Processing System. Vol. 23 (2010)
Van Hasselt, H., Guez, A. & Silver, D. Deep reinforcement learning with double Q-learning. In Proceedings of the 30th AAAI Conference Artificial Intelligence. Vol. 30 (2016)
Deng, Q., Santos, B. F. & Curran, R. A practical dynamic programming based methodology for aircraft maintenance check scheduling optimization. Eur. J. Oper. Res. 281(2), 256–273. https://doi.org/10.1016/j.ejor.2019.08.025 (2020).
doi: 10.1016/j.ejor.2019.08.025
ASA. Certification Specifications and Guidance Material for Master Minimum Equipment List (CS-MMEL). https://www.easa.europa.eu/document-library/certification-specifications/cs-mmel-issue-3 .