A visual experiment on 200 possible layouts discovering a 'best' 10 times better than worst.

The experiment showed how people choose a magazine and has set the magazine cover style since 2010. Actual results are reproduced below with kind permission of NewsCorp Australia.

The ProblemGardening Weekly is just one of over thirty magazine titles produced by News Corporation Australia but was failing in the market. In an effort to boost magazine sales, a redesign of the cover was proposed. The designer presented a variety of concepts that included elements from existing layout and website as well as some new ideas. Naturally there were many opinions about “what works” SurveyEngine were hired to help put some data into the discussion.​Initially, NewsCorp wanted to simply market test the handful of concepts produced by the artist but SurveyEngine proposed to test all of the combination of ideas to explain what elements drive a consumer’s choice.

How we did itThe artist produced four general ideas on revitalising the existing design.

MastHead – The designer suggested the masthead could be solid white as it had been or ‘bled-through.

Font Face – There were three possibilities – uppercase, lowercase and a cursive script style

​​Article Layout – Four options were presented – a side feature layout, two horizontal feature styles as well as ‘no features’

Background – Eight backgrounds were suggested – a number of flowers, a garden, a bowl of berries and a gardener.

These raw ideas, were developed as an experiment within SurveyEngine which reported that there were exactly twenty four test layouts that would be needed for the experiment. This list was given to the artist who produced finished artwork which were attached to the experiment.The experiment was sent out to 1,000 people appearing as a regular online survey. Once enough data had been collected, the results were automatically modelled.

Results

But what is the best layout?We can bypass all the mathematics and find the best layout by picking the most positive effects.

The Best

The optimal setting for each of the attributes is a:

see through masthead

simple uppercase font

background of a field of lavender

simple article layout

with a predicted choice 93% compared to worstThe cover on the left was created from the above optimal setting. Before the model, it had never existed and it was never directly tested, it was predicted.

The cover on the right was created synthetically from the above optimal setting. Before the model, it had never existed and it was never directly tested, it was predicted.

…and the worst​Predicted Choice 7% As an exercise, the worst layout was also created. This time all the most negative effects are picked, which are:

the solid masthead

the script font

the picture of a gardener

the side layout

with a predicted choice 7% compared to the bestThis means the optimal would be chosen between ten and twenty times more often than the worst.

When shown side by side, most people agree with the prediction. General comments were: “The best one seems more pleasing and simpler” or “I can imagine having this on my coffee table”.

It is worth reiterating that these covers had never existed before the model was generated, neither were they shown to any respondents in a traditional research sense. Rather, they were ‘found’ from hundreds of other possible alternatives.

What was implementedThis experiment set the course for all future versions of the Gardening Australia cover. The magazine tends to have flora on the cover rather than people unless they are celebrities. The design uses the see-through masthead and is uncluttered. ‘Gardening Australia Magazine’ images show how consistently the model results are adhered to since the experiment in 2010.NewsCorp did not implement all optimal model results. Commercial constraints meant the article layout needed to remain. Although, in this case they could use the model to avoid the least preferred option of the side layout. The font-face was not rigorously enforced by the model. The artists continue to choose their preferred font face. Although, the disliked ‘script font’ is conspicuously absent from recent covers.Having a model to understand choice means that informed decisions about the product can be made even if there are constraints.