The Infona portal uses cookies, i.e. strings of text saved by a browser on the user's device. The portal can access those files and use them to remember the user's data, such as their chosen settings (screen view, interface language, etc.), or their login data. By using the Infona portal the user accepts automatic saving and using this information for portal operation purposes. More information on the subject can be found in the Privacy Policy and Terms of Service. By closing this window the user confirms that they have read the information on cookie usage, and they accept the privacy policy and the way cookies are used by the portal. You can change the cookie settings in your browser.

K-Means Clustering Optimizing Deep Stacked Sparse Autoencoder

Source

Abstract

Because of the large structure and long training time, the development cycle of the common depth model is prolonged. How to speed up training is a problem deserving of study. In order to accelerate training, K-means clustering optimizing deep stacked sparse autoencoder (K-means sparse SAE) is presented in this paper. First, the input features are divided into K small subsets by K-means clustering, then each subset is input into corresponding autoencoder model for training, which only has fewer nodes in the hidden layer than traditional models. After training, each autoencoder’s trained weights and biases is merged to obtain the next layer’s input features by feedforward network. The above steps are repeated till the softmax layer, then fine-tuning is carried out. Using MNIST-Rotation datasets to train the network that has three hidden layers and each layer has 800 nodes, the improved model has higher classification accuracy and shorter training time when K = 10. With K increasing, the training time is reduced to almost the same as the fine-tuning time but the recognition ability is descended. Compared with the recently stacked denoising sparse autoencoder, the recognition accuracy is improved by 1%, not only the noise factor is not selected but also the training speed is significantly increased. The trained filters from the improved model is also used to train convolutional autoencoder, and it performs better than traditional models. We find that pre-training stage doesn’t need large samples simultaneously, and small samples parallel training reduces the probability of falling into the local minimum.

Accessibility options

You can change the active elements on the page (buttons and links) by pressing a combination of keys:

TAB go to the next element

SHIFT + TAB go to the previous element

Financed by the National Centre for Research and Development under grant No. SP/I/1/77065/10 by the strategic scientific research and experimental development program:
SYNAT - “Interdisciplinary System for Interactive Scientific and Scientific-Technical Information”.