An Evidence-Based Take on Engagement and Surveys: Interview with Paul Fairlie, Founder & CEO of Heliosophy

Paul Fairlie is the Founder/CEO of Heliosophy (www.heliosophy.ca). He is a consultant, researcher, speaker, and expert in work psychology with 20+ years of experience in organizational consulting, market research, and opinion polling. His approach to organizations is highly informed by evidence-based, positive organizational psychology. Paul is a behavioural scientist with a PhD in psychology, and foundations in organizational behaviour, personality theory, positive psychology, and advanced analytics.

Paul, you have devoted much of your career to studying and consulting on the science of engagement at work. In your opinion, why are organizations so interested in engagement?

It would only be an opinion. There are likely several reasons. One, engagement is one of the most unique variables to emerge from Organizational Behavior research in decades. We’re always looking for the next, new shiny object. Also, given the transformations that organizations have been going through, in response to globalization, changing demographics, increased competition, digital revolution, etc., leaders may feel the need to ‘up their game’ in terms of the employee experience in order to manage performance and retention. Then there’s the business press, which started popularizing the word ‘engagement’ in the late 1990’s, so it’s also pure marketing.

Indeed marketing efforts have created many management myths in the past. What are the top 3 misconceptions about engagement and engagement surveys?

One enduring myth is that there’s low agreement on how to define engagement. While that may be true of the professional world, but there’s actually pretty good agreement among work researchers. Another myth is that commercial engagement surveys necessarily measure engagement. They often don’t. Overall, I’d say the biggest ‘reveal’ about engagement is that it started off, years ago, as a research variable. The consulting industry then lifted ‘engagement’, as a term, to re-label and re-brand a lot of existing things. Old wine in new bottles. It’s ironic that while ‘real’ engagement has bigger impacts on organizations than these other things, the variable, itself, was passed over in favour of the mere word used to describe it!

So what is engagement, really?

The concept has evolved over time, but it was fine-tuned by researchers like Wilmar Schaufeli andArnold Bakker. It has cognitive aspects, such as attention (“I concentrate a great deal on my work”) and absorption (“Time flies on my job”), as well as emotional elements, such as energy or vigour (“I feel energetic when doing my job”). Some researchers add physical (“I really ‘throw’ myself into my job”) and dedication components. These things tend to stick together as a factor. Yes, there are quibbles among some researchers, but they only look bigger in the language of scientific discourse. In reality, they’re just nailing down the last 5 to 10 percent. And by the way, that doesn’t that engagement was invented by researchers. It was discovered through qualitative interviews with workers.

Is there a difference between engagement and job satisfaction?

Engagement has been confused with a lot of things – employee initiative, commitment, discretionary effort, positive affect, stay intentions, and job satisfaction. Yet, over 100 peer-reviewed research studies, using mathematical modeling, have shown these things to be separate from engagement. Here is a meta-analysis of 90 studies.Job satisfaction is considered to be an emotional state resulting from appraising one’s job.

What are the main drivers of engagement? Or in other words, what should I focus on if I want to increase my people’s engagement?

There are no ‘three things’ that drive engagement, which is a common theme in the biz media. We know that over 50 work factors have been linked to engagement across 300+ studies, some of which are reviewedhere. There are some relatively stronger drivers of engagement, such aswork-role fit. This is part of meaningful work, whichstudy afterstudy has strongly connected to engagement. Personality counts as well. Studieshere andhere suggest that engaged people are likely to be more extraverted, conscientious, emotionally stable, and proactive. So, the engagement problem is also an employee selection problem, to some degree.

The question everybody want an answer to: is engagement relevant for performance?

Engagement has been correlated with employee and organizational performance. But does it precede and cause higher performance? Causal modeling studies suggest that it might, although they’re based on cross-sectional data. For example, it precedesretention, which affects organizational performance. Myrecent review also shows engagement to be a precursor of employee health, another driver of performance. Engagement predicts employee performancehere and here, as well ascustomer loyalty. However, we need more longitudinal studies. For example, current engagement levels predict laterfinancial returns. By the way, all of these researchers measured evidence-based engagement, not the things that you find on most commercial ‘engagement surveys’. It’s also critical to point out that many things affect performance, and should be surveyed, if possible. When I taught university HRM courses, one of my final exam questions was a root cause analysis of a company suffering from lower employee performance. The answer key contained over 50 evidence-based causes of employee performance (students had to identify and explain only 10)!

What else should we focus on then?

Job satisfaction is one thing, because it can have bigger effects on commitment and stay intentions than engagement (see studyhere). In fact, ameta-analysis suggested that job satisfaction and commitment may be more critical for performance than engagement. We should also focus on things that drive employee outcomes other than engagement. For example,work-life balance is a very weak driver of engagement. In my own studyhere, work-life balance ranked 30 out of 37 as a correlate of engagement. Yet, it was the #1 correlate of physical and mental health. Things like satisfaction, commitment, effort, stay intentions, and well-being have their own drivers, and their own links to performance. All of this suggests that we should also use short, ‘pulse’ employee surveys only sparingly. Measure too narrowly, and you don’t know what you don’t know. An employee survey developed from best practices can ask 80 or questions in 12 to 14 minutes. That’s a lot in a little bit of time. By the way, words like ‘driver’, of course, imply causation. What we often mean is association or prediction. But, the industry loves that term.

Why do you think an evidence-based approach to engagement and surveys is needed?

Evidence = money (e.g., profit, GDP) and human progress (e.g., global well-being, quality of life). Period. We have to get better at discussing ROI when we talk to leaders. What lift or gain do we achieve when we stop making decisions based on opinions, biz media hearsay, and gut intuitions? Science, as one aspect of evidence-based management, was invented to help us circumvent hundreds ofbiases that affect human judgment. For example, I once had trouble convincing a consulting firm that their 360° feedback survey didn’t measure anything well. They argued that each client’s report, with its peaks and valleys, made sense to them. On the next go-round, I produced the reports based on random data and let them interpret them (but not for too long). They had a hard time believing what I’d done, because of the human tendency to see meaningful patterns in random data (i.e.,pareidolia). Once they were convinced that the results were imaginary, they paid me imaginary dollars until I re-did the work. 😉

It should have several evidence-based drivers, of several employee outcomes that we know from research impact employee and organizational performance. It also needs to be developed based on good measurement science. Some leaders spend millions of dollars on personality tests for hiring decisions. Yet, they’re fine with having their teen-aged son come into the office on the weekend to develop their company’s employee survey product (yes, I’ve seen it happen). It’s just sentences and a 7-point scale, right? But employee surveys are also a psychometric tool, subject to the same hundreds of cognitive and affective biases that affect ‘test-taking’. We have to get better at explaining why measurement rigour is important. For example, if you leave out even oneor two of ten evidence-based steps in designing an employee survey (or a structured interview, or a performance appraisal form), its effectiveness can be reduced to nearly zero.Many research examples show that even small changes in wording or punctuation can lead to drastically different results, and poor decisions. Overall,Sixty years of studies have shown that surveys, when done well, are more accurate than subjective ways of gathering data, like walk-about’s, focus groups, and social media.

That says a lot about survey content, in terms of wording and measures. What about how we conduct surveys?

That’s a whole other kettle of fish. You can have the best possible employee survey in the world, and then have its effectiveness leveled by what you do with it, or don’t do with it.Thankfully, there’s a huge volume of research on how to conduct effective employee survey projects from start to finish. For me, as a consultant, one of the biggest challenges around getting high-value survey results has been how the company markets the survey to employees. How far in advance, and how often is it promoted? How many marketing channels are being used? How do you show strong leadership buy-in? How will it help the company to realize its mission, vision, and strategy? How will employees benefit from the results (i.e., “what’s in it for me?”). How will employees be empowered to make changes based on the results? Who can employees talk to for more information (i.e., two-way dialogue)?

Apart from marketing, there’s data analysis. Do you dump a huge binder of bar charts in the middle of the boardroom table, or do you use more advanced analytics to mine and reduce the data, and identify a handful of things to prioritize for change? You won’t see those things in averages and frequencies. Then, there’s what’s done with the results. Some companies pay to conduct employee surveys, and then store the results in the circular file (also called the wastebasket). I sometimes ask a question on employee surveys about what the company did with the last survey’s results. If employees say ‘not much’ or ‘nothing’, the responses rate for the survey is usually quite low, along with engagement.

Well, thank you Paul for your view on engagement, employee surveys and organizational performance! I feel we are now better equipped to understand, choose and design surveys and engagement solutions that can bring along actual results. Thank you!

Thank you! This has been fun.

ScienceForWorkis an independent, non-profit foundation of evidence-based practitioners who want to #MakeWorkBetter.

Our mission is to provide leaders and decision-makers with trustworthy and actionable insights from behavioural science.

Did you like this interview? Share it with your network by clicking on the buttons below!

Follow us onLinkedIn andTwitter to receive the most trustworthy scientific research summarized in less than 1000 words!

Share

Lorenzo Gallì is an Evidence-Based Management enthusiast that seeks to spread awareness regarding the science behind human behavior at work. With a Master of Science in I/O Psychology from the University of Bologna (IT), Valencia (ES) and Maastricht (NL), he now works at Mercer as a Talent Management consultant. A former professional basketball player, he still enjoys playing some ball, reading comics and dancing salsa.

4 comments

There is no way to measure whether any survey is accurate or useful. The result is just data and leadership and management have been ignoring survey results for decades. We have a whole industry of experts talking about employee engagement and experience and new surveys come out nearly every week. New keynote speakers, authors and consultancies, all focused on collecting the data as if the data was a soultion. Evidenced-based employee survey studies are nothing more than opinion if the evidence isn’t supported by results. For over twenty years I have been conduction employee surevys that consist of a single question, just one. The difference is that the CEO is asking the questin and a third party is collecting the data. Every day one or more issues are approved and handed off to management for execution and announcement to employees. The culture, politics and silos are suspended. Blockers are elevated all the way to the CEO if necessary. As you might guess, very few go that far. Evidence based managemrnt assumes management is making decisions based on the data. Given all that we know about people, notwithstnding what level in the organizatin, data can and will always be processed based on the comfort of those most affected. When blocking employee suggestins is made more unfomfortable than implementing, then the evidence based measurement can be applied to the outcome. The input is of no value until acted upon, just ask the employees.

Thanks for this Jim! I also accepted your LinkedIn request. We agree that there’s a big problem with ‘what’s done with the data’. I briefly highlighted this near the end of the interview, but it’s hard to get into specifics, as you did, in a short interview. I’ve been there. “Survey done. Check box filled. Budget expended so we don’t lose it next year.” But, and I’m sure you’ll agree, that good measurement and data are still important in the ideal scenario of good follow-through (and it does happen). To give up on those things because of what’s often not done with them is to commit a nirvana fallacy (baby with the bath water, etc.). But, you may not agree that there is such a thing as good measurement (“There is no way to measure whether any survey is accurate…”). I can’t address that here. All I can do is point to an entire field of psychometric research. Tens of thousands of studies showing the accuracy of such tools in many ways…when used well. I can’t even keep up with the research in this area. It’s like asking us to defend physics in a blog. Of course, many leaders “don’t know what they don’t know” about this field. Very little of this research informs the development and use of employee surveys. And so, leaders come to equate bad surveys with surveys in general (i.e., category error). And don’t get me started on some ‘experts’, ‘keynote speakers’, and ‘authors’.

Employee engagement has been researched like no other subject dealing with business results. It has been made so needlessly complex that no one understands is except arguably the researchers. Where’s the research that cites empirical data proving that any approach has transformed engagement to the point it impacted business performance. There’s a world of difference between research and results. Here’s where everyone misses the boat. Surveys and the resulting data are the least impactful aspects of an EE/EX initiative. Begin with who is conducting the project, nearly 100% by HR. HR is in no position to force change on a resistant leadership. They represent only a single voice at the table. Despite decades of research, survey optimization and automation nothing has happened. Sadly, the employees know this thus the high degree of ambivalence. I have yet to find a single EE/EX service provider citing the business results of their work as a marketing tool. Nor are there any consultants basing their fees on the financial results. Far more data is ignored than acted upon almost certainly reducing engagement. Here’s what a simple 9-word survey can create when the CEO is asking the question, HR is not involved and a third party reporting to the CEO collects the data suspending the negative influences of culture, politics and silos. This was done with 75% of the fees based on the impact on the income statement solely from employees becoming engaged. If you cannot cite empirical evidence of business keep selling to HR. Without a proven process to follow supported by empirical results, the research is meaningless. https://chiefexecutive.net/employee-engagement-ceos-actually-listening/