On slide 30 of lecture 2, it says that in order to convert some NFA with epsilon transitions to one without, we only need to change S (starting states) and the delta function. Shouldn't F (finish states) be changed as we as well?

No, the manipulation of delta to delta' takes care of that.
Say that you have the transition q1 ->(a) q2 ->(eps) qf, where qf is an accepting state,
then the new automaton will have q1 ->(a) qf, and there is no need to add q2 to our accepting states.