The monk of the empty hand is all about using improvised weapons. However, it can receive the feat "Improvised Weapon Mastery" as a bonus feat only at lvl6.

The thing I'm not sure about is: is he considered proficient with improvised weapons before that? Else it means either he gets the -4 penalty on all attacks until lvl6, or he doesn't use any improvised weapon until lvl6, both of which seems to go against the purpose of the archetype.

The monk of the empty hand treats improvised weapons as if she were proficient in them, and can wield normal weapons as improvised weapons. This effectively has the same end effect as the Catch Off-Guard feat, but isn't a feat and thus doesn't use up a feat slot since it's a variant class ability. Thus, they do not take a nonproficient penalty when using improvised weapons. The text is not as clear as it could have been, I guess, but hopefully common sense can step in to bolster that if someone doesn't read this post?

(We COULD have simply said, "The monk of the empty hand gains "Catch Off-Guard" as a bonus feat, I guess, but that would have lost the flavor bit about how they often wield normal weapons as improvised weapons.)

Thus, you don't need to take Catch Off-Guard at all, not even for Improvised Weapon Mastery since you don't need to meet requirements for Monk bonus feats.

It seems like that is the case. There's nothing in the proficiency section that mentions proficiency with improvised weapons, and none of their other class features give proficiency with improvised weapons. This seems incredibly odd, given that they get a bunch of improvised weapon specific stuff before level 6, and they'll have a pretty tough time hitting anything with that -4 to attacks. Personally, I think it would make much more sense to just give the proficiency at level 1, but you'd have to talk to your DM about that.