The ever increasing number of surveillance camera networks being deployed all over the world has resulted in a high interest in the development of algorithms to automatically analyze the video footage, but has also opened new questions as how to efficiently manage the vast amount of information generated and, more important, how to protect the privacy of the individuals being recorded in their daily life. In this paper, we present a survey on video summarization techniques developed in order to efficiently access to the points of interest in the video footage. Thereby, we emphasize on the links that these techniques show with the task of privacy protection and draw lines of future research directions to incorporate indexing and summarization as tools for privacy protection by design.