Statistical Applications of Linear Programming for Feature Selection via Regularization Methods

by Yao, Yonggang

Abstract (Summary)
We consider statistical procedures for feature selection defined by a family of regularization problems with convex piecewise linear loss functions and penalties of L_1 or L_infinity nature. For example, quantile regression and support vector machines with L_1 norm penalty fall into the category. Computationally, the regularization problems are linear programming (LP) problems indexed by a single parameter, which are known as "parametric cost LP" or "parametric right-hand-side LP" in the optimization theory. Their solution paths can be generated with certain simplex algorithms. This work exploits the connection between the family of regularization methods and the parametric LP theory and lays out a general simplex algorithm and its variant for generating regularized solution paths for the feature selection problems. The significance of such algorithms is that they allow a complete exploration of the model space along the paths and provide a broad view of persistent features in the data. The implications of the general path-finding algorithms are outlined for various statistical procedures, and they are illustrated with numerical examples.
Bibliographical Information:


School:The Ohio State University

School Location:USA - Ohio

Source Type:Master's Thesis

Keywords:variable selection grouped statistical regularization parametric linear programming simplex algorithm tableau solution path support vector machine quantile regression dantzig selector functional component pursuit smoo


Date of Publication:01/01/2008

© 2009 All Rights Reserved.