Okay so I got too the second dungeon. The first one after the Rhombus dungeon. But it said I had too be level 18. I tried doing some side quests. For instance I tried doing this one goat side quest but, the enimies where too strong for me so I died instantly. Should I just go back too the begining area and complete all those quests then come back? Or will I be fine?

I mean it's certainly POSSIBLE, watch Xdragon's speedrun and he's basically starting the temple at level 4. But you're gonna have a bad time if you aren't already really good at the combat system, since most things in there will one or two-hit kill you. I'd strongly recommend heading back Rookie Harbor and Autumn's Rise and trying some of the earlier quests around your level, or even just picking fights around Autumn's Rise instead of jumping ahead to Bergen. Start with hedgehags, they're easy enough, and build from there once they become trivial.