It's simple mechanical logic.
Paper is a bunch of rough fibers snagged and tangled together.
When water worms it's way between any two, the rough edges separate, and fail to hold together.

On a larger scale, if two trapeze artists are gripping each others wrists, they stay together. If there's anything between them, like they are wearing gloves, the grip lessens. Something fat, like mittens, and they can no longer get a good grip at all.

OK, thanks, blank stare. That explains paper, but does the same logic apply to things like climbing rope, which are not held together by short, rough-edged fibers, but by long, continuous ones? I know rope doesn't get as weak as paper, but it does get weaker (by 10-15%, as I understand it.)

Paper is made up cellulose. Cellulose is a long chain of glucose molecules. Now, the thing about the molecular structure of cellulose is that adjacent chains of cellulose can form bonds with each other, called hydrogen bonds. This leads to the formation of fibres, and ultimately gives us paper.

Now, when paper is made wet, capillary action ensures that the water gets spread around in it. This water leads to a breakage of the inter-fibre hydrogen bonds, i.e., overall, the bonds that are keeping the paper together have weakened quite a bit. Apply enough pressure, and poof the paper tears.

When it comes to rope, you will notice that only the cotton-based ones weaken noticeably when wet. Why? Because cotton is mostly cellulose too. Wood gets soft when it has been in water for a long time. Again, because it is largely cellulose.