Inverse halftoning is a process to recover the multi-tone images from halftone images. Since halftoning is not an invertible process, there is no clear best or unique approach for inverse halftoning. In this paper, we propose a low complexity, high quality Fast Adaptive-spatial-varying Filtering (FAF) for inverse halftoning, which is robust to different kinds of halftone images. FAF is a two-step algorithm combining spatial-variant and spatial invariant filtering. Firstly, it uses spatial invariant filter to suppress the halftone noise and then generates the target image. Secondly, it uses spatial variant filter to filter the halftone image based on the target image. Combining these two filtering, FAF can filter the noise and preserve edges of the halftone image at the same time, which is essential for effective inverse halftoning.