Automatic, Efficient and Scalable Provenance Registration for FAIR HPC workflows
Version 1

Provenance registration is becoming more and more important, as we increase the size and number of experiments performed using computers. In particular, when provenance is recorded in HPC environments, it must be efficient and scalable. In this paper, we propose a provenance registration method for scientific workflows, efficient enough to run in supercomputers (thus, it could run in other environments with more relaxed restrictions, such as distributed ones). It also must be scalable in order to deal with large workflows, that are more typically used in HPC. We also target transparency for the user, shielding them from having to specify how provenance must be recorded. We implement our design using the COMPSs programming model as a Workflow Management System (WfMS) and use RO-Crate as a well-established specification to record and publish provenance. Experiments are provided, demonstrating the run time efficiency and scalability of our solution.


Filename: COMPSs Provenance WORKS 22.pdf  Download

Format: PDF document

Size: 2.75 MB

help Creators and Submitter

Views: 609   Downloads: 51

Created: 2nd Aug 2023 at 15:53

Last updated: 7th Nov 2023 at 11:35

help Attributions


Version History

Version 1 (earliest) Created 2nd Aug 2023 at 15:53 by Raül Sirvent

No revision comments

Powered by
Copyright © 2008 - 2024 The University of Manchester and HITS gGmbH