Without taking a close look: frame-type decision only happens in first pass. Depending on the settings that can take significant cpu time (relative to the rest of the time spent). I see you are using --bframes 5 with --b-adapt 2.