A&S Graduate Studies

Location

Abstract:
The machine learning (ML) system has been an indispensable part of the ML ecosystem in recent years. The rapid growth of ML brings new system challenges such as the need of handling more large-scale data and computation, the requirements for higher execution performance, and lower resource usage, stimulating the demand for improving ML system. General-purpose system optimization is widely used but brings limited benefits because ML applications vary in execution behaviors based on their algorithms, input data, and configurations. It’s difficult to perform comprehensive ML system optimizations without application specific information. Therefore, domain-specific optimization, a method that optimizes particular types of ML applications based on their unique characteristics, is necessary for advanced ML systems. This dissertation performs domain-specific system optimizations for three important ML applications: graph-based applications, SGD-based applications, and Python-based applications.

For graph-based applications, this dissertation introduces ATMem, a runtime framework to optimize application data placement on heterogeneous memory systems. ATMem aims to maximize the fast memory (small-capacity) utilization by placing only critical data regions that yield the highest performance gains on the fast memory. It applies an adaptive data chunk profiling method with a lightweight sampling-based profiler, and identifies the critical data chunks of graph data structures by applying a local relative ranking strategy and an m-ary tree-based method. Additionally, ATMem employs a multi-stage multi-threaded migration mechanism to enable high-bandwidth data migration and reduce TLB misses. Experimental results show that ATMem achieves significant speedup over the baseline that places all data on slow memory (large-capacity) with only placing a minority portion of the data on fast memory.

For SGD-based applications, this dissertation proposes a lossy compression scheme for application checkpoint constructions (called LC-Checkpoint). LC-Checkpoint intends to simultaneously maximize compression rate of checkpoints and reduce recovery cost of SGD-based training processes. LC-Checkpoint applies an exponent- based quantization and a priority promotion operation on saving the most critical information for SGD recovering to checkpoints and uses a Huffman coding to leverage the non-uniform distribution of the gradient scales. Extensive experiments show that LC-Checkpoint achieves a high compression rate with a lower recovery cost over a state-of-the-art algorithm.

For Python-based applications, this dissertation develops PieProf, a lightweight profiler to detect application interaction inefficiencies. The goal of PieProf is to measure the inefficiencies in the native execution and associate inefficiencies with high-level Python code to provide a holistic view for diagnosing interaction
inefficiencies. PieProf leverages Chabbi-Liu Algorithm, an efficient method to detect redundant memory access, in a complicated multi-languages environment. Guided by PieProf, 17 real-world applications are optimized, yielding numerous nontrivial speedups.

The ongoing research project aims to design and implement a domain-specific software package for solving large-scale kernel principal component regression problems.

Bio:
Yu Chen is a Ph.D. candidate in the Department of Computer Science at William & Mary. He is co-advised by Dr. Bin Ren, Dr. Zhenming Liu, and Dr. Andreas Stathopoulos. His research lies in machine learning systems, with a focus on building profiling tools and performing system-algorithm co-design to optimize machine learning applications. His Ph.D. research appeared in CGO 2020, ICML 2020, and FSE 2021. Previously, he received his Bachelor of Software Engineering at Southeast University, China, in 2014. Prior to his Ph.D., he worked as a senior software engineer in the industry.

This Event Appears On

• W&M Featured Events
• A&S Graduate Studies