Ok, I took some time to test how it works on real hardware. i were right about VSRAM addresses $4E and $4C being used for vertical scrolling of the left-most column when partially scrolled horizontally, but it's not exactly as I thought.

Here are the conclusions of my tests in column vscroll mode:

1) When fine horizontal scrolling value is applied (hscroll % 16 != 0), the left-most column is partially displayed and the vertical scroll value applied is VSRAM[$4C] & VSRAM[$4E]. The same value is applied for both A & B planes.

2) The above statement is only true in 40-cell mode. In 32-cell mode, the left-most column cannot be scrolled vertically, i.e vscroll value is fixed to zero for both planes. I verified that writing vscroll value to other VSRAM addresses had no effects as well.

Those offsets are the two last entries of VSRAM ($4c is the one used for the last column of plane A and $4e the one used for the last column of plane B), one column being 2 cells (16 pixels).

What I imagine is that, when hscroll value is not exactly a multiple of 16 pixels, the left-most column, which is only partially displayed, is rendered earlier and vertical scroll is not parsed correctly because it's not column 0 yet (technically, it would be column -1). Maybe in this case it would use the last value that was read on previous line (in this case, vscroll of last column for the previous line), I'm not sure why it get the same value for both planes though neither why the two last values get anded or why it does not happen in 32-cell mode.