I do have them all from 1959-1968 and can say definitively that the 1967 MLB portion of that set (150 total cards) is the most difficult to assemble. 1959 had been very tough until recently when Larry Sarver discovered a contact in Caracas who consigns lower grade cards to him on the regular. He has had quite a few of the high numbers available over the past couple years.

1968 has a couple cards that are extremely rare for unknown reasons. The 3rd checklist in the 1962 set seems to be the toughest of that set. No true card scarcities exist in 1960, 1964 & 1966.

If the goal was just to build a set in any condition, then you are absolutely correct, your best bet would be 1964 & 1966. While they may seem like less of a challenge compared to the rest, those are both monster sets of 370, and if you wanted only cards without evidence of glue or other common major flaws (writing, rat-gnawed corners, tape), I suspect it would be a bear to build either set from scratch right now with better than an average grade of VG.

Personally, I think the overall "easiest" of the sets to build in decent condition is 1960. The cards don't have glue problems. The stars are limited, the Mantle/Boyer & Yaz are super tough because of the competition for rare star items, but even they can be found occasionally. Hardly anyone is working on one, so there is almost no competition or market for commons.

My favorite set of all of them is the 1967 full set. I think it is absolutely spectacular. The Venezuelan league cards are mostly gorgeous, some have amazing backdrops. The Retirado cards are as unique of a creation the hobby has ever seen. And the MLB portion is shrouded with mystique and unbelievable scarcity for a post-war issue.