You can also imagine that there are other elements inside the scollable div and that they should stay clickable even if they are masked by the margin of the "drag me" element (which should be the case when using CSS margins).

I'm looking preferably for a CSS-only solution that works in Webkit browsers.

The easiest way to achieve this is to create an invisible CSS ::before pseudo-element that covers the box plus a padding, and to make it transparent to the mouse interactions using the pointer-events property: