In the 2003–04 season, Görlitz made the transition to defender, and appeared in 32 matches for TSV, scoring his first professional goal on 27 March 2004, at Eintracht Frankfurt, through a header (3–0 triumph), as his side was finally relegated, after ranking 17th.

In the summer of 2007, it was announced that Görlitz would join newly promoted Karlsruher SC, on loan for the 2007–08 season.[2] The player only missed three league matches and his club finished 11th, well clear from relegation.

On 9 May 2008, the loan deal was extended for another season.[3] Following his second year, in which he featured significantly less, also suffering another relegation, Görlitz returned to Bayern, to play through the final year in his contract.[4] For the third consecutive season at the club, he was often demoted to the reserve squad.