To make your rivers a little straighter, I suggest rotations of the d4 be to the second-next clockwise face that doesn't currently have a path on it (instead of just the next clockwise face, as per when you're creating paths).

I also suggest that any time you either generate something that looks absurd or that has 3+ streams flowing into a single hex, you make it a little pond or lake. Hexes adjacent to water-filled hexes count as having one stream flowing into them across the adjacent face if you're rolling for rivers. This will give you a handful of lakes and ponds of varying sizes.

I come from Canada, which has most (in the sense of a slight absolute majority) of the world's lakes, so I always think fantasy maps don't have enough open bodies of water of significant size on them, but this is a bit of an idiosyncrasy. If you do this procedure a few times across the length of the map, you'll eventually end up with a nice hydrological basin with rivers and lakes all connected up.