A&S Graduate Studies
[PAST EVENT] Yu Chen, Computer Science - PhD Defense
Abstract:
The machine learning (ML) system has been an indispensable part of the ML ecosystem in recent years. The rapid growth of ML brings new system challenges such as the need of handling more large-scale data and computation, the requirements for higher execution performance, and lower resource usage, stimulating the demand for improving ML system. General-purpose system optimization is widely used but brings limited benefits because ML applications vary in execution behaviors based on their algorithms, input data, and configurations. It’s difficult to perform comprehensive ML system optimizations without application specific information. Therefore, domain-specific optimization, a method that optimizes particular types of ML applications based on their unique characteristics, is necessary for advanced ML systems. This dissertation performs domain-specific system optimizations for three important ML applications: graph-based applications, SGD-based applications, and Python-based applications.
For SGD-based applications, this dissertation proposes a lossy compression scheme for application checkpoint constructions (called LC-Checkpoint). LC-Checkpoint intends to simultaneously maximize the compression rate of checkpoints and reduce the recovery cost of SGD-based training processes. Extensive experiments show that LC-Checkpoint achieves a high compression rate with a lower recovery cost over a state-of-the-art algorithm. For kernel regression applications, this dissertation designs and implements a parallel software that targets to handle million-scale datasets. The software is evaluated on two million-scale downstream applications (i.e., equity return forecasting problem on the US stock dataset, and image classification problem on the ImageNet dataset) to demonstrate its efficacy and efficiency. For graph-based applications, this dissertation introduces ATMem, a runtime framework to optimize application data placement on heterogeneous memory systems. ATMem aims to maximize the fast memory (small-capacity) utilization by placing only critical data regions that yield the highest performance gains on the fast memory. Experimental results show that ATMem achieves significant speedup over the baseline that places all data on slow memory (large-capacity) with only placing a minority portion of the data on fast memory.
The future research direction is to adapt ML algorithms for software systems/architectures, deeply bind the design of ML algorithms to the implementation of ML systems, to achieve optimal solutions for ML applications.
Bio:
Yu Chen is a Ph.D. candidate in the Department of Computer Science at William & Mary. He is co-advised by Dr. Bin Ren, Dr. Zhenming Liu, and Dr. Andreas Stathopoulos. His research lies in machine learning systems, with a focus on building profiling tools and performing system-algorithm co-design to optimize machine learning applications. His Ph.D. research appeared in CGO 2020, ICML 2020, and FSE 2021. Previously, he received his Bachelor of Software Engineering at Southeast University, China, in 2014. Prior to his Ph.D., he worked as a senior software engineer in the industry.