The pending $700 million sale of a 25,000-square-foot retail site on Fifth Avenue this week was driven by ever higher rents, growing tourist traffic and tight property supply on the renowned street, according to one of the buyers and brokers.

The site at East 55th Street is a half-block long and sits smack in the middle of Fifth Avenue's highest-priced stretch—roughly between East 51st and East 57th, depending on who is counting.