From January 1921 to mid-autumn Pervukhin worked as the Executive Secretary of the Proletarian Thought. He was a member of the Bureau of the Zlatoust Komsomol District Committee, and later became the head of its Department for Political Education in April 1922. Later that year he became the Zlatoust Komsomol District Committee's Deputy Secretary, and was its Technical Secretary from April to August 1922.[1]

The Metal Workers' Union of the Zlatoust District Committee ordered Pervukhin to Moscow in the late summer of 1922 to study. He graduated in 1929 from the Electrical Department of the Plekhanov Moscow Institute of the National Economy with a degree in electrical engineering. Following his graduation, Pervukhin started work at Mosenergo, the Moscow electric power company. In May 1936 he became the Director of the Kashirskaya Power Plant. From June to September 1937, Pervukhin worked as Mosenergo's Chief Engineer, and later that year became its acting head. Pervukhin started to work for the People's Commissariat for Heavy Industry in late 1937, and was later appointed to the post of Deputy People's Commissar for Heavy Industry in 1938, and First Deputy People's Commissars for Heavy Industry in June 1937 when Lazar Kaganovich was People's Commissar for Heavy Industry.[1] During the Great Purge Pervukhin was promoted to Deputy Head of the Moscow Electrical Power Administration Bureau, and then its head.[2] On 24 January 1939 Pervukin was promoted to the newly established post of People's Commissar for Electric Power Stations and was given a seat in the Communist Party's Central Committee at the 18th Party Congress.[1]

Pervukhin opposed Khrushchev's Regional Economic Soviet reform, whose main aim was to reduce the powers and functions of the central ministries. He told Khrushchev and other Presidium members that this reform would weaken branch administration, and that the centralisation and specialisation which had been the system's cornerstone would be lost. Instead, Pervukhin proposed to reduce the numbers of central ministries and establish territorial commissions to provide "horizontal cooperation".[10] Later, in 1957, Pervukhin joined the Anti-Party Group in a bid to remove Khrushchev as First Secretary.[11]

Following the failed bid to remove Khrushchev, Pervukhin was demoted to a non-voting member of the Presidium, and became the Soviet Union's ambassador to East Germany in 1958.[1] As ambassador, Pervukhin observed that "the presence in Berlin of an open and essentially uncontrolled border between the socialist and capitalist worlds unwittingly prompts the population to make a comparison between both parts of the city, which unfortunately, does not always turn out in favor of the Democratic [East] Berlin".[12] Pervukin remained wary, until its very creation, of establishing a sectorial barrier between East and West Berlin; he believed that creating a barrier would increase anti-Soviet sympathies not only in Berlin but in Germany as well. Instead, he proposed three options: 1) "introducing restrictive measures" for East Germans to enter both East Berlin and West Berlin; 2) strengthening the border security; 3) stopping the free movement between the two cities.[13] However, he did admit that closing the borders was a possibility, claiming that if the political situation worsened, the East German regime and the Soviets would not have another option.[14]

Walter Ulbricht, the East German leader, invited Pervukhin to his summer house to discuss the East German immigration flow to West Germany. There Ulbricht told Pervukhin that if the Soviets did not react soon, East Germany would "collapse".[15] Pervukhin discussed other problems as well, claiming that Ulbricht but also the East German leadership in general, were opposed to the Soviet Union's plan to improve relations with West Germany.[16] When Khrushchev gave his approval to construct what would become the Berlin Wall, Pervukhin was the first to know.[17] Ulbricht told Pervukhin of the need to create the East–West barrier at night, and he and Khrushchev would later agree to this.[18]