The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop:ISAV 2020: In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization
Authors: Seif Ibarhim, Cyrus Harrison, and Matthew Larsen (Lawrence Livermore National Laboratory)
Abstract: Derived field calculations are a vital part of the visualization and analysis workflow. These calculations allow simulation users to create important quantities of interest that are not generated by the simulation, and systems that calculate derived quantities must be flexible enough to accommodate a wide variety of user requests. In situ analysis imposes additional constraints on the system, and derived field calculations must be able to leverage the same resources as the simulation does to minimize the runtime and memory usage. Just-in-time (JIT) compilation defers code creation until runtime, and a JIT based system is capable of fusing a complex expression into a single kernel invocation (i.e., kernel fusion). Without kernel fusion, the system would be forced to evaluate each piece of the expression (e.g., an operator or function call) as a separate kernel invocation, which increases both runtime and memory pressure on the host simulation. In this paper, we present a production-oriented in situ derived field system that leverages JIT compilation to target heterogeneous HPC architectures. Additionally, we explore the runtime costs of using this system to calculate three expressions in three simulation codes.