Bailey is well known for research involving biology and sexual orientation. In the early 1990s Bailey and Richard Pillard coauthored a series of twin studies which examined the rate of concordance of sexual identity among monozygotic twins (52% concordance), dizygotic twins of the same sex (22%), non-twin siblings of the same sex, and adoptive siblings of the same sex (11%).[10][11] More recent research by Bailey et al. on twins however found much lower concordance rates for monozygotic twins regarding homosexual orientation of only 20% for men and 24% for women pointing to a significant contribution of environmental factors in sexual orientation; Bailey suggests an explanation for the much lower concordance rate among monozygotic twins in this study as opposed to previous studies: In those previous studies, twins deciding whether to participate in a study clearly related to homosexuality probably considered the sexual orientation of their co-twins before agreeing to participate.[12]

Another line of Bailey's research has concerned the ways that homosexuals are sex-atypical (or gender nonconforming) compared with heterosexuals, as well as the ways that homosexuals are sex-typical and gender conforming. For example, he published a meta-analysis showing that on average, homosexual men and women recall being much more gender nonconforming children, compared with heterosexual children.[13] In contrast, he also showed that for many traits related to mating (such as interest in casual sex, and emphasis on a partner's physical attractiveness), homosexuals appear to be similar to heterosexuals of their own sex.[14] Recently, he has researched the gaydar phenomenon.[15]

Bailey has been interested in the evolutionary paradox of the persistence of homosexuality. "Male homosexuality is evolutionarily maladaptive," he told The New York Times which also noted that Bailey intended "that the phrase means only that genes favoring homosexuality cannot be favored by evolution if fewer such genes reach the next generation."[16]

In an article coauthored with Aaron Greenberg, he suggested that allowing parents to choose the sexual orientation of their children is morally acceptable, provided the means used to accomplished that goal are themselves morally acceptable.[17] (For example, killing infants who will become homosexual would obviously be wrong. The acceptability of aborting "gay fetuses" or "straight fetuses" would depend on whether one believed that abortion, per se, is morally acceptable.) Alice Dreger criticized Greenberg's and Bailey's argument[18] and they responded.[19]

A third line of research has examined sexual arousal patterns and their relation to sexual orientation in men and women. This research has focused on both genital and self-reported sexual arousal measures. For example, Bailey's lab showed that men's genital sexual arousal patterns closely tracked their sexual orientations, but women's did not.[20] In 2005 this research produced a study which questioned whether male bisexuality exists in the way that it is sometimes described; the study was based on results of penile plethysmograph testing. The testing found that of men who identified themselves as bisexual, 75% were only aroused genitally by homosexual imagery, and 25% were only aroused genitally by heterosexual imagery. They concluded: "Male bisexuality appears primarily to represent a style of interpreting or reporting sexual arousal rather than a distinct pattern of genital sexual arousal."[21] The study received wide attention after a New York Times piece on the study.[22] A 2011 study using similar methodology filtered participants more stringently, requiring at least two sexual partners of each sex and at least one romantic relationship lasting three months or longer; this study finds both genital and subjective arousal, though it is not clear which arousal pattern is more prevalent in the modern bisexual community.[23][24]

The 2005 article and study were criticized by gay and bisexual groups[25] and by FAIR.[26] Critics argued the sample size was relatively small, consisting of one hundred (100) men. Also, all of these subjects were "self-selected", from ads placed in gay and "alternative" publications. Then the researchers had to disregard results of thirty-five percent (35%) of this population, as non-responders.[21] Agreeing with the author's conclusion that bisexuality is a subjective experience, the late Fritz Klein, a sex researcher and the author of The Bisexual Option argued that "social and emotional attraction are very important elements in bisexual attraction."[22]

Bailey's sexual arousal work was attacked by The Washington Times and some socially conservative commentators as prurient and a waste of taxpayer dollars.[27] Bailey[28] and his defenders[29] have responded that studying sexual arousal patterns is important in understanding human sexuality, especially sexual orientation. Recently, Bailey's lab has begun to study sexual orientation and sexual arousal using fMRI.[30]

Bailey's book The Man Who Would Be Queen: The Science of Gender Bending and Transsexualism was published in 2003.[31] In it, Bailey reviewed evidence that male homosexuality is innate, a result of heredity and prenatal environment. He also reviewed the theory of Ray Blanchard that there are two unrelated forms of transsexualism, one that is an extreme type of homosexuality and one that is an expression of a paraphilia known as autogynephilia. Written in a popular science style, the book summarized research supporting Bailey's opinions.

The book generated considerable controversy. The most detailed investigation into that controversy was reported by Alice Dreger,[32] a bioethicist and historian, known for her activism in support of intersex rights. Dreger included additional details in Galileo’s Middle Finger, an analysis of modern clashes between scientists and activists whose beliefs are challenged by them.[33] In her documented account of the Bailey case, she concluded that a small group of self-styled activists tried to bury a politically challenging scientific theory by attacking Bailey. "These critics, rather than restrict themselves to the argument over the ideas, had charged Bailey with a whole host of serious crimes," but that "what they claimed about Bailey simply wasn't true."[34]

A transgender woman that he described in the book filed a complaint with Northwestern University alleging that her many discussions with Bailey about his view of trans women and the book he was writing made her a non-consensual subject of IRB-regulated research by Bailey, and that during this time, she had consensual sex with him.[35] Northwestern found no basis for the complaint.[32][36] Transgender professors Lynn Conway and Deirdre McCloskey filed a complaint against Bailey with Illinois state regulators, alleging that he practiced psychology without a license by providing brief case evaluation letters suggesting candidacy for sex reassignment surgery; however, the department did not pursue those allegations, as he did not accept remuneration for the services and therefore did not violate the law.[32][37] At least two women who said they were subjects in his book filed a complaint with Northwestern alleging that Bailey committed scientific misconduct by not informing them that they were to be the subjects of research used in the writing of his book.[38][39] Northwestern did investigate this allegation. Although the findings of that investigation were not released,[40] Northwestern's Vice President for Research, C. Bradley Moore, said, "The allegations of scientific misconduct made against Professor J. Michael Bailey do not fall under the federal definition of scientific misconduct."[32] and that the university "has established a protocol to help ensure that Professor Bailey's research activities involving human subjects are conducted in accordance with the expectations of the University, the regulations and guidelines established by the federal government and with generally accepted research standards."[32] Bailey says that he did nothing wrong and that the attacks on him were motivated by the desire to suppress discussion of the book's ideas about transsexualism, especially autogynephilia.[41]Alice Dreger, a bioethicist, published an account of the controversy in the Archives of Sexual Behavior.[32] According to Dreger, the allegations of misconduct could accurately be described as "harassment",[42] and an "anti-Bailey campaign".[32] Dreger wrote that of the four women who complained to Northwestern, two acknowledged that they were aware they would be included in Bailey's book in their letter to the university. The other two were not described in the book. Dreger also reported that while there was no definitive evidence to refute the allegation of sexual misconduct, datestamps on e-mails between Bailey and his ex-wife indicated that he was at her home looking after their two children at the time the misconduct was said to have occurred. The journal published in the same issue 23 commentaries regarding multiple aspects of the controversy, including criticism of Dreger's analysis.[43]

Outside of the transsexual community and sexology researchers, this controversy is largely notable because of its implications for academic freedom and freedom of speech. In an interview with The New York Times, Dreger said, "If we're going to have research at all, then we're going to have people saying unpopular things, and if this is what happens to them, then we've got problems not only for science but free expression itself."[37] While Conway compared his work to Nazi propaganda, and Andrea James posted pictures of his children on her website with sexually explicit captions, other critics believe that their actions against Bailey and his book represent legitimate comment on a topic of public interest.[44]

Helen Boyd explained what might have motivated some to object to the book:[45]

In the crossdressing community, the man who admits he is turned on by his dressing is still considered a pervert. The autogynephilic transsexual will not receive the same sympathy for her transsexualism as the non-autogynephilic transsexual. That's exactly what makes Bailey's book so dangerous: it allows transsexual women to be condemned by our society for having "perverse" sexual arousal patterns.

— Helen Boyd

In response to such criticisms, Bailey reiterated a line from his book: "True acceptance of the transgendered requires that we truly understand who they are."[46]

Bailey and his work were featured prominently in a Boston Globe story by Neil Swidey entitled "What Makes People Gay?"[47] That story was included in the 2006 volume of "The Best American Science Writing."[48]

Bailey and his lab were also prominent in the CBS News60 Minutes story "Gay or Straight?,"[49] which first aired on March 12, 2006 and was the most popular news story on the CBS News website the following week.[50] This story provoked reactions ranging from "fantastic and fascinating" by gay author Jeremy Hooper at the Good As You blog[51] to negative comments from gay author David Ehrenstein, who noted the show was "replete with the sort of clichés about gay men and effeminacy that haven’t been seen in a network news context since the 1967 CBS broadcast The Homosexuals."[52] Shari Finkelstein, the producer of the "Gay or Straight" segment, responded: "We were aware of the controversy surrounding Michael Bailey's book, and we looked into all the allegations before we decided to interview him for the story...." She concluded: "We didn't feel there was anything that disqualified him from being interviewed. And in fact, his work is highly regarded by all of the researchers in the field who we spoke with, and we felt that he was a very worthy person to discuss these issues."[53]

In 2011, Bailey's human sexuality class at Northwestern made the headlines of major news organizations after he allowed a female guest speaker and her male partner to perform an impromptu live mechanized sex toy demonstration using a "fucksaw"[54][55]—a modified reciprocating saw[56] converted into a sex toy by attaching a "phallic object" instead of a blade[57][58]—to bring the woman to orgasm in front the audience.[55][56] Students were advised beforehand of the nature of the demonstration in this optional after-class event on kinky sex and female orgasm.[57][58] In the aftermath, Northwestern University President Morton Schapiro criticized Bailey for "extremely poor judgment" and launched an investigation.[55][59] Bailey at first defended the demonstration, saying that students found lectures featuring guest speakers valuable, but subsequently issued an apology, saying he regretted the upset caused and its effect on the university's reputation.[57] He said there would be no repeats,[57] but maintained that the demonstration had been relevant to the topic of his course, and said that the students who chose to attend were over 18, "legally capable of voting, enlisting in the military, and consuming pornography", and contended that the criticism he had received was poorly reasoned.[57] The response among academics was mixed. Joseph Epstein criticized Bailey's class as failing academic standards in a long piece for The Weekly Standard, and ultimately compared Bailey to a pimp.[60] In contrast, Laurie Essig, writing in The Chronicle of Higher Education, thought that the incident "triggered a national conversation about what we can and cannot look at".[61] In a web-only feature for Esquire, Bailey's former research assistant Paul Schrodt defended his teaching and research methods.[62] Alice Dreger also defended Bailey's class as being of high quality in general, but agreed with Shapiro that the demonstration "was a case of poor judgment, because it wasn't worth it".[63] Eventually, in response to the incident, Northwestern administrators removed Bailey's human sexuality course from the following year's curriculum.[64] A year later, Northwestern reintroduced a somewhat differently themed sexuality class taught by Lane Fenrich, according to whom the new class emphasized "the major questions" and "the major thinkers" rather than being "geared toward sexual practices".[65]

^Carey, Benedict (2007-08-21). "Criticism of a Gender Theory, and a Scientist Under Siege". New York Times. Retrieved 2008-06-26. "What happened to Bailey is important, because the harassment was so extraordinarily bad and because it could happen to any researcher in the field," said Alice Dreger, an ethics scholar and patients’ rights advocate at Northwestern who, after conducting a lengthy investigation of Dr. Bailey's actions, has concluded that he is essentially blameless.

^Carey, Benedict (2007-08-21). "Criticism of a Gender Theory, and a Scientist Under Siege". New York Times. Retrieved 2008-06-26. "Nothing we have done, I believe, and certainly nothing I have done, overstepped any boundaries of fair comment on a book and an author who stepped into the public arena with enthusiasm to deliver a false and unscientific and politically damaging opinion," Deirdre McCloskey, a professor of economics, history, English, and communication at the University of Illinois at Chicago, and one of Dr. Bailey's principal critics, said in an e-mail message.