Enhancing Agile Big Data Project Success using Project Management Body of Knowledge (PMBOK) Performance Domain

Abstract

Big data projects, especially within the fields of data science, data analytics, and data engineering, are growing rapidly.  This growth can be seen by machine learning technologies and the emerging trend of generative AI that utilize large datasets as input.  This rise is evidenced by trend reports from leading IT companies, which indicate significant growth in the use of big data.  The prevailing trend in big data projects is inconsistent with their actual execution.  A considerable proportion of big data initiatives fail to reach the production phase due to inherent challenges, as these projects often display agile characteristics owing to their rapid pace and fluctuating requirements, in accordance with industry trends and needs.  This highlights the need to assess the issues faced in agile big data projects.  A thorough literature review was performed to identify issues, thus leading to the formulation of suggestions grounded in the PMBOK 7th edition as the standard and guideline for project management.  The SLR phase effectively identifies four main categories of challenges: human resources, project management, data and information management, and organizational issues.  The subsequent recommendation tackles these challenges.  This study utilizes seven of the eight performance domains outlined in the PMBOK 7th edition to address the identified difficulties.