Hm, thanks, but re-re-re-checked and checked again when you said it is wrong, I have no idea what might be wrong. It is wired excactly like it is described in the tutorial and I double checked with the picture also, just to be sure. The ATtiny2313 is not upside down. I don't know what to do different. I guess the resistors would only prevent damage but not solve my problem, right?

I was finally able to upload the Blink sketch to the ATtiny2313! I tried Ernis suggestion and moved it to a different breadboard and it worked right away. I feel kind of stupid for not thinking of trying that on my own.Thanks so much for your help, guys.