I'm fast running out of I/O on my micro and can't spare 8 pins - so this
means I need to do something smart to lower my pin counts.

Been looking at the 74HC165 and 74HC166 which are 8 bit
parallel-in/serial-out shift register.

The difference between them is that:

165 is asynchronous
166 is synchronous

What I want to do is:
1. get the 8 bit data from the encoder
2. send it to the micro
3. during the sending, the encoder data could change, but I want it ignored
until all the data is sent.
4. when sending has completed, new data is then latched into the register
again and the process is repeated.

So my question is, what's the difference between the 165 and 166 and more
importantly, which one will do what I described above?

For your application it doesn't really matter.
The "Synchronous" device will only load during a clock transition (when
PL is enabled), useful when you cascade devices and want to sync serial
data for example. You don't need this.
The "asynchronous" device will load any time the PL input pin
transitions, this is all you need. So all you need do is toggle the PL
pin to latch the data and then shift with the clock input. 3 lines
required, PL/CLK/DATA. Input changes will be ignored durign shifting
while PL is high.

Yes, easy peasy, but I forgot to mention...
Because you are sampling asynchronous data, there may be times when
your input changes while you are latching the data. In this case you
may get garbled data.

Your inputs must stay stable and meet the "setup time" and "hold time"
of the chip (probably around 10ns or so, the datasheet will tell you)
while it is being latched. This is not unique to the 165/166 it is the
same for any latch, even if you feed the inputs straight into the
micro.
Just be wary that this could happen. If you see any "weird" data, this
is what it could be.
The usual fix for this in software is to just ensure that you read say
two or three samples the same one after the other before you register
that data as being ok.

Before you ask, no, you can't use the "synchronous" 166 to fix this
problem :->