Less than you would at the top of the skyscraper or less than you would on a beach at the same elevation?

The universal law of gravitation applies to all masses, not just when one of the masses is the Earth. The skyscraper has mass, you have mass, therefore the there is a gravitational attraction between you and the mass of the skyscraper above you.

That's why I ask the question "compared to what?" Considering the magnitude of the masses involved, changing your elevation relative to the center of the Earth would have more of an effect than the mass of a mere skyscraper above you.