Get stronger in the main compound lifts (dips are one of the best) for now. If you get stronger then you're arms will definitely grow, and once you've got some solid strength you can start to bring back in direct arm work.