Intrepydd: Toward Performance, Productivity, and Portability for Massive Heterogeneous Parallelism
This paper introduces our ongoing work on the automatic ahead-of-time (AOT) parallelization of Python programs on recent and future hardware systems with massive parallelism and heterogeneity. Our approach is driven by the combination of ML-based type prediction and multi-versioned code generation that guarantees the correctness of our type-specific code optimizations in all cases. While Python is a dynamically-typed language, recent research demonstrated it is highly possible to predict what data types are likely to occur at runtime, by ML-based static prediction and/or runtime type profiling in numerical computation kernels. Given code fragments with predicted data type information, our optimization engine performs automatic parallelization and sophisticated high-level code optimizations for the target system, such as shared/distributed heterogeneous hardware platforms. Our approach introduces novel extensions to the polyhedral compilation to integrate loop and data layout transformations as well as automated selection of CPU vs. GPU code variants. Our preliminary empirical evaluation shows significant performance improvements relative to sequential Python in both single-node and multi-node experiments.
Mon 21 OctDisplayed time zone: Pacific Time (US & Canada) change
11:00 - 12:30 | Pre-lunch SessionVIVEKFEST at Pacific C Chair(s): Rajiv Gupta University of California at Riverside (UCR) | ||
11:00 20mResearch paper | Intrepydd: Toward Performance, Productivity, and Portability for Massive Heterogeneous Parallelism VIVEKFEST Jun Shirako Georgia Institute of Technology, Tong Zhou Georgia Institute of Technology, Akihiro Hayashi Georgia Institute of Technology | ||
11:20 20mResearch paper | Verification of Concurrent Programs Using Hybrid Concrete-Symbolic Interpretation VIVEKFEST | ||
11:40 10mTalk | A Few Lessons and Problems For Life; Source@Vivek Sarkar VIVEKFEST V Krishna Nandivada IIT Madras | ||
11:50 20mResearch paper | Evaluation of Speedup & Energy with Multigrain Parallelizing Compiler VIVEKFEST John Pickar , Tohma Kawasumi , Hiroki Mikami Waseda University, Japan, Keiji Kimura Waseda University; Japan, Hironori Kasahara Waseda University, Japan | ||
12:10 20mResearch paper | A Formal Model for Portable, Heterogeneous Accelerator Programming VIVEKFEST |