The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop:DRBSD-6: The 6th International Workshop on Data Analysis and Reduction for Big Scientific Data
Authors: Manish Parashar (Rutgers University)
Abstract: Extreme-scale scientific workflows continue to be challenged by the massive amounts of data that needs to be exchanged between workflow components and the associated costs. While in-situ workflow formulations are addressing some of these challenges and are enabling the creation of extreme scale coupled simulations and analysis workflows, these workflows involve complex data-driven interactions and data exchange patterns and require complex cost-performance tradeoffs and runtime resource management. In this talk, I will explore these challenges and investigate how AI/ML advances and be used to enable intelligent staging-based data management. This research is part of the DataSpaces project.