Pair new professors with a more senior faculty member (more than 2 years but before 'retirement mode') with built in activities such as:

-doing observations of each other and then discussing the results

Schedule observations of professors in other departments who exemplify effective teaching for new professors to visit

Plan for 3 rounds of student evaluations per semester to get a more holistic perspective of new professors (Mid-course, End of Course, Next semester) - along with that I would do more of a 360 in general - triangulation improves data validity (peer feedback, student evaluations, personal reflections, student work samples, etc) but use it as a support system not simply an evaluation tool

Request student work samples (beyond just tests/grades) thereby providing documentation that there is a relationship between what is being taught and what students know and are able to do -it holds both parties accountable (teacher and student) - and it's been my experience that (transparency, communication and accountability are all necessary elements in an effective learning/development continuum), - I would have this 360 conversation by semester with either the mentor or the Chair (Example)

Create a forum for both novice and seasoned professors where they can share "their favorite project(s)" those they think are yielding the most effective results with their students - have this "show and tell" every semester - I'd recommend doing this university-wide

More collaboration in general - whether in the departments themselves like PLC's but I would recommend experimenting with theAction Learningmodel - it's a different way of analyzing challenges.

Never doubt that a small group of thoughtful, committed people can change the world. Indeed, it is the only thing that ever has.