Scalable Workflow-Driven Hydrologic Analysis in HydroFrame
Document Type
Conference Proceeding
Publication Date
2020
Abstract
The HydroFrame project is a community platform designed to facilitate integrated hydrologic modeling across the US. As a part of HydroFrame, we seek to design innovative workflow solutions that create pathways to enable hydrologic analysis for three target user groups: the modeler, the analyzer, and the domain science educator. We present the initial progress on the HydroFrame community platform using an automated Kepler workflow. This workflow performs end-to-end hydrology simulations involving data ingestion, preprocessing, analysis, modeling, and visualization. We demonstrate how different modules of the workflow can be reused and repurposed for the three target user groups. The Kepler workflow ensures complete reproducibility through a built-in provenance framework that collects workflow specific parameters, software versions, and hardware system configuration. In addition, we aim to optimize the utilization of large-scale computational resources to adjust to the needs of all three user groups. Towards this goal, we present a design that leverages provenance data and machine learning techniques to predict performance and forecast failures using an automatic performance collection component of the pipeline.
Publication Information
Purawat, Shweta; Olschanowsky, Cathie; Condon, Laura E.; Maxwell, Reed; and Altintas, Ilkay. (2020). "Scalable Workflow-Driven Hydrologic Analysis in HydroFrame". In V.V. Krzhizhanovskaya, G. Závodszky, M. H. Lees, J.J. Dongarra, P.M.A. Sloot, S. Brissos, and J. Teixeira (Eds.), Computational Science: ICCS 2020: (Lecture Notes in Computer Science series, Vol. 12137, pp. 276-289). https://doi.org/10.1007/978-3-030-50371-0_20
Comments
Computational Science: ICCS 2020 is volume 12137 of the Lecture Notes in Computer Science book series.