To summarize chips last post (and kick us off here), here are the concerns Chip would like to address:

1) When assigning pre-defined custom menus to a Theme's defined Theme Locations, the process now requires considerably more steps, clicks, page refreshes, and time

2) When looking at the "Select a menu" dropdown, there is no way to determine if all Theme locations have an associated custom menu assigned

3) When editing a given menu, in the Theme Location field checkboxes, there is no indication that a given Theme Location already has a custom menu assigned to it

4) With long-ish custom menus, the "Theme Locations" field is buried "beneath the fold", resulting in no initially visible UI for assigning the current menu to a Theme Location

5) Overall, the page now seems to emphasize adding/editing custom menus, and seems to deemphasize assigning custom menus to Theme Locations, and the latter is arguably the more important role/task for Appearance -> Menus

1) When assigning pre-defined custom menus to a Theme's defined Theme Locations, the process now requires considerably more steps, clicks, page refreshes, and time

I agree, this does take a bit more time. Again, there is no perfect scenario here. I see 5 options. each has pros and cons, but we have to pick one of them.

1) Meta box - left column at the top

PRO's

It's in your face, you can't miss it

When switching themes, you can update all of your theme locations in one spot.

It's familiar for existing users

CON's

It's unnecessarily in your face. ;-)

It pushes menu item options down the page

It's awkwardly grouped in that left column when everything else is a menu item option

It forces users to take a third step (outside the flow of creating the menu) at the end of creating a menu, where they then have to select a theme location.

2) Meta box - left column at the bottom

PRO's

When switching themes, you can update all of your theme locations in one spot.

CON's

Chances are, it would get pushed past the fold for some users

It's awkwardly grouped in that left column when everything else is a menu item option

It forces users to take a third step (outside the flow of creating the menu) at the end of creating a menu, where they then have to select a theme location.

3) Meta box - right column at the top

PRO's

It's in your face, you can't miss it

When switching themes, you can update all of your theme locations in one spot.

CON's

It's unnecessarily in your face. ;-)

It would through the balance of the page all off

It pushes the menu editing screen down the page

It's awkwardly grouped in that right column when everything else is related to editing a specific menu

It forces users to take a third step (outside the flow of creating the menu) at the end of creating a menu, where they then have to select a theme location.

4) Meta box - right column at the bottom

PRO's

When switching themes, you can update all of your theme locations in one spot.

CON's

We actually tested this placement and it failed, because people were clicking the "theme location" "save" button incorrectly thinking that it was saving the changes to their menu.

Chances are, it would get pushed past the fold for some users

It's awkwardly grouped in that right column when everything else is related to editing a specific menu

It forces users to take a third step (outside the flow of creating the menu) at the end of creating a menu, where they then have to select a theme location.

5) A checkbox for each theme location inline for each menu

PRO's

It removes the "theme locations" meta box from the left column, making the actions one takes in that column nice and clear. This also pulled the menu item options up, making them more visible.

It reduced the visual clutter on the page. Forcing the user (especially a new user) to grok what was going on when they were presented with options to add menus, and edit menus, and add menu items, and set theme locations all on the same UI, was just too much to ask. For you it makes sense sure. It's the way you've been doing it all along. But for most users it was a terribly confusing experience.

It reduces the steps (page reloads) to get set up with a new menu from 3 down to 2.

CON's

We're moving functionality that existing users will have grown used to.

When changing themes, you've got to hop into each individual them to set the theme location.

Weighing my options, I'd be comfortable with either 5, or 2.

2) When looking at the "Select a menu" dropdown, there is no way to determine if all Theme locations have an associated custom menu assigned

All of the theme locations are listed out under each menu. When you load the page, the menu you edited most recently will be auto loaded. If another menu is set as a theme location, you will see it in grey next to the theme location:

So, yes, you will always see at-a-glance which theme locations have menus assigned.

3) When editing a given menu, in the Theme Location field checkboxes, there is no indication that a given Theme Location already has a custom menu assigned to it

Not true. See my answer in 2.

4) With long-ish custom menus, the "Theme Locations" field is buried "beneath the fold", resulting in no initially visible UI for assigning the current menu to a Theme Location

You're right. Again, it comes down to weighing the options out of the 5 listed above.

5) Overall, the page now seems to emphasize adding/editing custom menus, and seems to deemphasize assigning custom menus to Theme Locations, and the latter is arguably the more important role/task for Appearance -> Menus

I agree that we are emphasizing adding/editing custom menus. You can't set a menu to a theme location if you can't figure out how to create a menu in the first place. ;-)

My worry with this would be, how often do you customize menus? My guess is that many (dare I say most) users will do it once to add menus while first customizing their site, and then only very occasionally revisit to make small changes.

But the resultant UI places all prominence on creation/editing of menus, rather on the more-frequent activity of assigning menus to Theme Locations.

The two primary was that custom menus are used after creation are:

Assigning them to Theme Locations

Assigning them to Widgets

Number 2 is a different configuration screen, and thus off topic.

That leaves number 1. And I must disagree that having the number one most-frequent use of a custom menu "in your face" is a "CON".

This new UI inverts the relationship between custom menu and Theme Location. Under the previous UI, and as designed/intended, custom menus are assigned to Theme Locations. However, under the new UI, Theme Locations are assigned to custom menus.

But the resultant UI places all prominence on creation/editing of menus, rather on the more-frequent activity of assigning menus to Theme Locations.

The two primary was that custom menus are used after creation are:

Assigning them to Theme Locations

Assigning them to Widgets

Number 2 is a different configuration screen, and thus off topic.

That leaves number 1. And I must disagree that having the number one most-frequent use of a custom menu "in your face" is a "CON".

Do you have some kind of data to support this assertion that changing theme locations is somehow a more frequent task than editing menus? For regular, non-power-users?

This new UI inverts the relationship between custom menu and Theme Location. Under the previous UI, and as designed/intended, custom menus are assigned to Theme Locations. However, under the new UI, Theme Locations are assigned to custom menus.

Yes, yes it does.

The theme location drop down allows you to select from multiple menus, which tells me it belongs on the menu management screen, not the add/edit screen (which should be all about a single menu).

And yet, the resultant UI puts the Theme Location inside the metabox for individual menus - which is, essentially, exactly what you were arguing against earlier.

There are a couple of things in play here:

Global vs individual menu settings. Before, you had a mix of global and individual settings that required multiple steps and experienced knowledge to configure. Adding & managing menus and setting locations were global, editing the menu title and items, auto-add pages were individual. Now, editing the menu title, items, location and auto-add pages are individual. The concept of adding a new menu was merged with the menu-editing UX and menu management stayed global but became more discoverable.

The 1:1 vs 1:many locations paradigm. With menus, there's always been (and still is) a 1:many relationship. You can set one menu to many locations. And with locations, it's 1:1. One location, one menu. The previous UI put emphasis on the locations side of the paradigm and completely ignored the menu side, which was odd because it was a menu-specific setting in a global context. The UI change makes that menu-specific setting individual rather than global.

An average number of Theme switches greater than 1 will all but guarantee that users assign custom menus to Theme Locations more often than they create menus.

This new UI inverts the relationship between custom menu and Theme Location. Under the previous UI, and as designed/intended, custom menus are assigned to Theme Locations. However, under the new UI, Theme Locations are assigned to custom menus.

Yes, yes it does.

The theme location drop down allows you to select from multiple menus, which tells me it belongs on the menu management screen, not the add/edit screen (which should be all about a single menu).

And yet, the resultant UI puts the Theme Location inside the metabox for individual menus - which is, essentially, exactly what you were arguing against earlier.

There are a couple of things in play here:

Global vs individual menu settings. Before, you had a mix of global and individual settings that required multiple steps and experienced knowledge to configure. Adding & managing menus and setting locations were global, editing the menu title and items, auto-add pages were individual. Now, editing the menu title, items, location and auto-add pages are individual. The concept of adding a new menu was merged with the menu-editing UX and menu management stayed global but became more discoverable.

The 1:1 vs 1:many locations paradigm. With menus, there's always been (and still is) a 1:many relationship. You can set one menu to many locations. And with locations, it's 1:1. One location, one menu. The previous UI put emphasis on the locations side of the paradigm and completely ignored the menu side, which was odd because it was a menu-specific setting in a global context. The UI change makes that menu-specific setting individual rather than global.

Interestingly, reading through the other ticket, it seems that more people liked the tabbed interface, and that the tabbed interface tested the best. But (without any supporting UI testing), one person opined that a tabbed interface was "too heavy" and the idea was subsequently dropped.

Regarding the 1:1 vs 1:many relationship: the previous Theme Locations metabox met that need perfectly, and remains better than the current implementation in that regard. It was very easy to see:

All the available Theme locations

Menus assigned to all available Theme locations

The same menu potentially assigned to multiple Theme locations

The ability to change one or all Theme Location menu assignments, in one step

User testing means testing the same workflow (script) across all tests in order to do comparisons. So, yes, it would be helpful for you to test a different script/workflow and give your feedback based on trying out the different flows, even if you're coming in with preferences already. Not sure what being involved in UI has to do with it - user testers certainly are not involved in WP core UI.

This is tricky. I feel like the theme locations as checkboxes works best with our set of testing scenarios. However, as listed above, there are some downfalls to making the switch:

We're moving functionality that existing users will have grown used to.

When changing themes, you've got to hop into each individual them to set the theme location.

When existing users with many menu items, load the page, their theme locations setting will be gone (pushed down below the fold).

Is theme locations as checkboxes enough of an improvement to warrant these downsides?

But, I can't see us keeping the theme location metabox in the top left position. 3 out of 3 of the users we've tested experienced the same difficulties with this layout (the first user in this round, and ​both users here).

I think there's another upside to moving the location settings into the same area as the auto-add pages option: The settings are grouped and if we were to add the nav_menu_settings action proposed in #18584, that section would be extensible.

With the locations meta box, menu-specific settings were scattered all over the UI. This change brings them all together in the menu editor where they intrinsically belong.