Discussion: A tale of three cousins: Lasso, L2Boosting and Dantzig
N. Meinshausen, G. Rocha, B. Yu · The Annals of Statistics · 2007
We would like to congratulate the authors for their thought-provoking and interesting paper.The Dantzig paper is on the timely topic of high-dimensional data modeling that has been the center of much research lately and where many exciting results have been obtained.It also falls in the very hot area at the interface of statistics and optimization: ℓ 1 -constrained minimization in linear models for computationally efficient model selection, or sparse model estimation (Chen, Donoho and Saunders [5] and Tibshirani [17]).The sparsity consideration indicates a trend in high-dimensional data modeling advancing from prediction, the hallmark of machine learning, to sparsity-a proxy for interpretability.This trend has been greatly fueled by the participation of statisticians in machine learning research.In particular, Lasso (Tibshirani [17]) is the focus of many sparsity studies in terms of both theoretical analysis (Knight and Fu [10], Greenshtein and Ritov [9], van de Geer [19], Bunea, Tsybakov and Wegkamp [3], Meinshausen and Bühlmann [13], Zhao and Yu [23] and Wainwright [20]) and fast algorithm development (Osborne, Presnell and Turlach [15] and Efron et al. [8]).Given n units of data Z i = (X i , Y i ) with Y i ∈ R and X T i ∈ R p for i = 1, . . ., n, let Y = (Y 1 , . . ., Y n ) T ∈ R n be the continuous vector response variable and X = (X 1 , . . ., X n ) T the n × p design matrix and let the columns of X be normalized to have ℓ 2 -norm 1.It is often useful to assume a linear regression model,