At a meeting of the Party in Munich in September 1919, the main speaker was Gottfried Feder. When he had finished speaking, a member of the audience whose name is lost to history stood up and suggested that Bavaria should break away from Prussia and form a separate nation with Austria. Adolf Hitler sprang up from the audience to rebut the argument. Drexler approached Hitler and thrust a booklet into his hand. It was entitled My Political Awakening and, according to Hitler's writing in Mein Kampf, it reflected much of what he had himself decided upon. Later the same day he received a postcard telling him that he had been accepted for membership of what was at that time the German Workers' Party. After some internal debate, he says, he decided to join.

By 1921, Hitler was rapidly becoming the undisputed leader of the Party. In the summer of that year he travelled to Berlin to address a meeting of German Nationalists from northern Germany. While he was away the other members of the Party Committee, led by Drexler, circulated as a pamphlet an indictment of Hitler, which accused him of seeking personal power without regard to other considerations. Hitler brought a libel suit and Drexler was forced to repudiate at a public meeting. He was thereafter moved to the purely symbolic position of honorary president, and left the Party in 1923.

Drexler was also a member of a völkisch political club for affluent members of Munich society known as the Thule Society. His membership in the NSDAP ended when it was temporarily outlawed in 1923 following the Beer Hall Putsch, in which Drexler had not taken part. In 1924 he was elected to the Bavarian state parliament for another party, in which he served as vice-president until 1928. He had no part in the NSDAP's refounding in 1925, and rejoined only after Hitler had come to power in 1933. He received the party's "Blood Order" in 1934 and was still occasionally used as a propaganda tool until about 1937, but was never again allowed any real power. He was largely forgotten by the time of his death.

He is survived today by one grand nephew (George Drexler), one grand niece (Liba Drexler), several great grand nieces and nephews (Freidrich, Rebecca, Josef, Jesse and Krista Drexler) and great great grand nieces and nephews (Timothy, Mika and Hans Drexler — Timothy being heir to the family breweries and Drexler Manor).