Phase Transitions for Clustering, Signal Recovery and Hypothesis Testing
Speaker:Prof. Wanjie WANG
Assistant Professor
National University of Singapore
Date & Time:14 Dec 2016 (Wednesday) 10:00 - 11:00
Venue:E11- 1040
Organized by:Department of Mathematics

Abstract

We consider a two-class clustering problem, where we observe a large number. of features but only a small fraction of them contribute to the class labels. There are three problems here: 1) Test whether there are two clusters or not; 2) Recover the useful features; 3) Identify class label for each sample. In the two-dimensional phase space calibrating the rarity and strengths ofuseful features, we find the precise demarcation for the Region of Impossibility and Region of Possibility for these three problems. In the former, useful features are too rare/weak for successful clustering. In the latter, useful features are strong enough to allow the problem solved successfully. We also propose two PCA methods and two aggregation methods. In the region of possibility, one or more of these four methods yield satisfactory results.

Key words: High dimensional data; clustering; hypothesis testing; signal recovery; phase transition

Biography

Prof. Wanjie Wang obtained her bachelor degree at Peking University in 2009 and PhD at Carnegie Mellon University in 2014. Prof. Wang is now an assistant professor at National University of Singapore. Her research fields include high dimensional statistics, signal processing and random matrix.