The short answer to why you “have to” return a pointer is that you clearly don't have to return anything. You are not returning pointer (but an int) and you are not using the returned value for anything. You can change function's return type to void and remove the line with return and it will work just as well.

The reason why it works without returning anything is that you are passing a pointer to the array a as argument to sort and then modifying the array — known as x inside that function – in place. So there is only one array, and thus you don't have to return another one, or a pointer thereto, for the changes to be visible in main.