The problem is that until these “free floating rationales” for rejecting your siblings advances were realized, early (and I mean early) man may have been doing his sister without thinking twice about it. As you stated above, “most sexual organisms are highly motivated to engage in intercourse with no awareness of the connection between that act and reproduction“, and as groups were starting to form, it would seem that there would be a high demand for available mates. I’m also guessing that some of these early “successful organisms” were getting it on with their siblings merely because they were “highly motivated“ to do so, and there would have really been no reason not to. Early man obviously didn’t seem to be very chatty, as we have no historical knowledge or written information of their activities until they evolved to about 7,000 bc so, although one group may have been communicating in some fashion with those within their groups that incest was unacceptable, other groups may not have had any concerns about possible consequences until later in history or when communication between groups became more common. And, as I stated earlier, it should be noted that it is quite a leap from our basic knowledge of humans being only those fossil finds we run across to a relatively highly advanced society on the evolutionary timeline at around 7,000 bc.

Perhaps the place to turn for a discussion on this is cultural anthropology. Some idea of cultural continuity is found by looking for parallels/commonalities in social rules and customs in various civilizations and tribes. This would be especially helpful in terms of sex/reproduction and traditions surrounding family units. Although not an exact process, anthropologists can formulate some ideas about how human culture evolved, extrapolating into the past divergent lines of traditions, taboos, etc. Some of the same assumptions are used when studying languages.