Recently, data mining algorithms running on large-scale problems face a trade-off between computing time and accuracy. It is hard to find an algorithm which reduces computing time without sacrificing accuracy. Due to wide application area, research on alternating direction method of multipliers (ADMM) has attracted huge interest. On L1-regularized least squares regression problem, specifically on an application to hyperspectral imaging, we empirically show that ADMM has outstanding performance in both computing speed and accuracy. In particular, we test the effect of increasing penalties on the performance of ADMM on both synthetic and real-life problems; and in our numerical experiments, we compare a variable penalty ADMM (VP-ADMM) algorithm with split Bregman method (SBM).
A hyperspectral image is a three dimensional data-cube in which the first two dimensions describe pixels in the 2D image and the third dimension records the electromagnetic reflectance of the corresponding pixel under varying wavelengths. Due to finite resolution, each pixel is composed of different materials which can be identified by exploiting their electromagnetic reflectance under varying wavelengths. Since most of the materials have tiny proportion in the combination, they can be safely omitted. We call the other ones which are dominant as endmembers. Therefore, for a fixed pixel, the data along the third dimension can be considered as a linear combination of the spectra of the endmembers of that pixel. Since only a few materials are dominant, the data-cube is quite sparse in the number of mixing endmembers with positive weight. The inverse problem of weight resolution can be modeled as an L1 regularized least squares problem. Numerical results show that the time spent for VP-ADMM to obtain similar resolutions is just 20% of that for SBM.
In conclusion, we demonstrate that VP-ADMM is superior on solving both random and real-life large scale problems. In particular, the empirical results for synthetic random experiments show that VP-ADMM can save up to 30% of the computing time when compared to SBM.