Maturity Assessment Model for Industrial Data Pipelines
2023 (English)In: Proceedings of the 2023 30th Asia-Pacific software engineering conference, apsec 2023, IEEE Computer Society Digital Library, 2023, p. 503-513Conference paper, Published paper (Refereed)
Abstract [en]
Data pipelines can be defined as a complex chain of interconnected activities that starts with a data source and ends in a data sink. They can process data in multiple formats from various data sources with minimal human intervention, speed up data life cycle operations, and enhance productivity in data-driven organizations. As a result, companies place a high value on strengthening the maturity of their data pipelines. The available literature, on the other hand, is significantly insufficient in terms of providing a comprehensive roadmap to guide companies in assessing the maturity of their data pipelines. Therefore, this case study focuses on developing a data pipeline maturity assessment model that can evaluate the maturity of data pipelines in a staged manner from maturity level 1 to maturity level 5. We conducted empirical research in order to develop the maturity assessment model on the basis of five different determinants to address the specific needs of each data pipeline maturity level. Accordingly, it aims to support organizations in assessing their current data pipeline maturity, determining challenges at each stage, and preparing an extensive roadmap and suggestions for data pipeline maturity improvement. In future work, we plan to employ the maturity model in different companies as a case study to evaluate its applicability and usefulness.
Place, publisher, year, edition, pages
IEEE Computer Society Digital Library, 2023. p. 503-513
Series
Asia-Pacific Software Engineering Conference, ISSN 1530-1362
Keywords [en]
Data Pipelines, Maturity assessment, determinants, factors, roadblocks, challenges, benefits, recommendations, roadmap, data pipeline maturity improvement
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:mau:diva-69984DOI: 10.1109/APSEC60848.2023.00062ISI: 001207000500053Scopus ID: 2-s2.0-85190526342ISBN: 979-8-3503-4417-2 (print)OAI: oai:DiVA.org:mau-69984DiVA, id: diva2:1886152
Conference
30th Asia-Pacific Software Engineering Conference (APSEC), DEC 04-07, 2023, Seoul, SOUTH KOREA
2024-07-302024-07-302024-07-30Bibliographically approved