Ultracold atoms in optical lattices are a versatile tool to investigate fundamental properties of quantum many body systems. In a series of experiments performed at the Max-Planck Institute for Quantum Optics in Garching, we demonstrated how the control of such systems can be extended down to the most fundamental level of single atomic spins at specific lattice sites. Using a high-resolution optical imaging system, we were able to obtain fluorescence images of strongly interacting bosonic Mott insulators with single-atom and single-site resolution [1] and addressed the atomic spins with sub-diffraction-limited resolution [2]. In addition, we directly monitored the tunneling quantum dynamics of single atoms in the lattice, and observed quantum-correlated particle-hole pairs [3] spreading of correlations after a parameter quench [4], and the quantum dynamics of spin-impurities [5]. Our results open the path to a wide range of novel experiments from observation of strongly correlated fermionic systems, implementation of novel cooling schemes, and engineering of quantum many-body phases to quantum information processing.