A&S Graduate Studies
[PAST EVENT] [via Zoom] Shuangquan Wang, Computer Science - Ph.D. Defense
Location
ZoomAbstract:
Unhealthy dietary habits (such as eating disorders, eating too fast, excessive energy intake, and chewing side preference) are major causes of some chronic diseases, including obesity, heart disease, digestive system disease, and diabetes. Dietary monitoring is necessary and important for patients to change their unhealthy diet and eating habits. However, the existing monitoring methods are either intrusive or not accurate enough. In this dissertation, we present our efforts to use wearable motion sensors to sense mastication dynamics for continuous dietary monitoring.
First, we study how to detect a subject's eating activity and count the number of chews. We observe that during eating the mastication muscles contract and hence bulge to some degree. This bulge of the mastication muscles has the same frequency as chewing. These observations motivate us to detect eating activity and count chews through attaching a triaxial accelerometer on the temporalis. The proposed method does not record any private personal information (audio, video, etc.). Because the accelerometer is embedded into a headband, this method is comparatively less intrusive for the user's daily living than previously-used methods. Experiments are conducted and the results are promising. For eating activity detection, the average accuracy and F-score of five classifiers are 94.4% and 87.2%, respectively, in 10-fold cross-validation test using only 5 seconds of acceleration data. For chew counts, the average error rate of four users is 12.2%.
Second, we study how to recognize different food types. We observe that each type of food has its own intrinsic properties, such as hardness, elasticity, fracturability, adhesiveness, and size, which result in different mastication dynamics. Accordingly, we propose to use wearable motion sensors to sense mastication dynamics and infer food types. We specifically define six mastication dynamics parameters to represent these food properties. They are chewing speed, the number of chews, chewing time, chewing force, chewing cycle duration, and skull vibration. We embed motion sensors in a headband worn over the temporalis muscles to sense mastication dynamics accurately and less intrusively than other methods. In addition, we extract 37 hand-crafted features from each chewing sequence to explicitly characterize the mastication dynamics using motion sensor data. A real-world evaluation dataset of 11 food categories (20 types of food in total) is collected from 15 human subjects. The average recognition accuracy reaches 74.3%. The highest recognition accuracy for a single subject is up to 86.7%.
Third, we study how to detect chewing sides. We observe that the temporalis bulge and skull vibration of the chewing side are different from those of the non-chewing side. This observation motivates us to deploy motion sensors on the left and right temporalis muscles to detect chewing sides. We utilize a heuristic-rules based method to exclude non-chewing data and segment each chew accurately. Then, the relative difference series of the left and right sensors are calculated to characterize the difference of muscle bulge and skull vibration between the chewing side and the non-chewing side. To accurately detect chewing sides, we train a two-class classifier using long short-term memory (LSTM), an artificial recurrent neural network that is especially suitable for temporal data with unequal input lengths. A real-world evaluation dataset of eight food types is collected from eight human subjects. The average detection accuracy reaches 84.8%. The highest detection accuracy for a single subject is up to 97.4%.
Bio:
Shuangquan Wang has been working on his second Ph.D. degree in the Department of Computer Science at William & Mary since Fall 2016. He is working with Dr. Gang Zhou in the fields of wearable & mobile computing, smart health, and human activity recognition. Shuangquan Wang received his first Ph.D. degree from Shanghai Jiao Tong University in 2008, an M.E. degree from Wuhan University of Technology in 2004, and a B.E. degree from Wuhan Institute of Technology in 2000.