You have to try and keep your Main Character level higher than your Dungeon Level.

You need to have at least the same number of guards as your dungeon level.

So, if you have a level 10 Dungeon, you need to have your main character at least at level 10 in the main class, and you need to have at least 10 guards.

If you do not have the both of the above, you will get a Loot Penalty, which means that your Dungeon Loot level is reduced until you correct the items above.

You can tell if you have a Loot Penalty by looking at the Loot tab on your dungeon information screen. It will be written there in red letters that a penalty is in place, and your adjusted Loot Level is also displayed.