Before I get to it, allow me to quote something from his Chofetz Chaim biography. But first I will preface it with Jacob Katz's famous quip that if he would be called by the name of his book, like a traditional talmid chochom, then he'd be known as the Shabbos Goy. It seems this is an old joke. On pg. 105 Yoshor writes:

It is related that Shumor, writer of Yiddish novels, once introduced himself to a well-known rabbi as an author. "So you are an author! What kind of books do you write?" Thereupon Shumor drew from his bosom his latest work, entitled The Defiant Renegade (Meshumad Le'haachis). The rabbi glanced at him askance. "Well, well, my dear man, you have created a queer monument for yourself. As you know, it is customary among Jews to call the author after the name of his book, also to inscribe the adopted name on his tombstone. Thus we have: 'Here rests the Magen Avraham, the Pene Yehoshua, the Peri Megadim, etc.' Whereas your tombstone will bear the inscription 'Here Rests the Defiant Renegade.'"

Yoshor adds a footnote where he related that R. Jacob Sapir (author of Even Sapir) was said to have been quarelling with Peretz Smolenskin, who had written a book called Kevuras Chamor ("The Burial of the Ass") and Sapir told him "Peretz, here is the difference between you and me. On my tombstone they're going to write 'Here lies the Even Sapir,' but on yours they will right 'Here is the Burial of an Ass.' The reason why he brings these up is of course to explain how a rabbi is supposed to embody the book he is identified with.

As an aside, the name of the actual short story is Mumar Le-hach'is, not Meshumad Le-hach'is (link). amusingly, the version I linked to on Google Books - from the Princeton Library - has stamps indicating that its first home was Rabbi Isaac Elhanan Theological Seminary - see the first page.)

In any event, here is his article on Yeshivas, which appeared in the pages of a periodical called Ha-mesilah in Vol. 5 May-June 1940 (link.) This issue was dedicated to Yeshivos, because of the ravages of the war in Europe. This article is not particularly historically illuminating, but it includes a paen to Torah and Derech Eretz, where Yoshor takes the position that in the view of the Sages, yeshiva students are engaged in a life of Torah and Derech Eretz (for they are termed 'builders.') He also quotes verse from Bialik.

That's what he means. He is countering those critics of yeshiva students who view them as cloistered, by pointing out that in the view of Chazal talmidei chachomim are "builders" and cultivators of society.

You are probably right that he is trying to give that impression. But he is very clever in not actually saying it.

It could also be understood that he's just taking it for granted that people are going to work for a living, and he just needs to explain that those workers who are also "wasting" a lot of their time with Torah are really "builders"