Digital marketers have always had some concerns with survey data. But they still rely on it for critical information, like, “Is our target audience even aware of our product? Do they ‘get’ the value prop? Why didn’t they download the app?” Certain questions can’t be answered in any other way.
So when survey respondents start to clam up or get suddenly “shy” — something we are seeing right now, in response to press surrounding vulnerabilities in data privacy — we should pay close attention and take radical action. Specifically, we have to be very clear with survey respondents about how we will protect their survey data. If we fail to do this, we will forever lose access to this critical stream of data.
Here’s why this is such a big deal. We conducted the exact same survey last year and this year. The surveys were specifically designed to understand just how open and honest people are willing to be in a survey. Or, put differently, just how freaked out people are getting about sharing their data. Even when survey questions come from a trustworthy source, and even when respondents remain anonymous, people still have trepidation. And while last year that trepidation was lukewarm, this year it turned white hot.
Both years, we recruited large samples of people via Google Surveys (5,582 in May 2017 and 6,298 in April, right after Facebook CEO Mark Zuckerberg’s hearings) and asked them the same 10 questions, which were designed to assess openness and honesty. Across the board, our 2018 respondents are proving overwhelmingly more jaded. Concerningly so.
To begin with, among those we recruited, fewer people agreed to complete the survey. In 2017, 18.7 percent of recruited respondents agreed to complete the survey. But that percentage dropped by an alarming 13 percent to just 16.6 percent in 2018. In 2018, we also saw a 19 percent drop in the amount of content provided in response to open-ended questions and 23 percent more refusal when it came to offering up personal information like household income. (All these results are statistically significant at the 95 percent confidence level.) In short, people don’t want to take surveys anymore, and when they do, they seem to be breezing through as much as possible and seeking out the quickest exits.
In 2018, we also saw higher indicators of lying in survey responses, based on measures adapted from standard psychological tests of lying. For example, we saw a 10 percent climb in people stating that they “always” practice what they preach and a 12 percent uptick in people saying that they are “always” as honest as possible (both measures of “socially desirable” lying). So, along with the slackening response this year, there was also a significant hike in untrustworthiness of responses. People aren’t so sure they want to reveal much anymore.
What’s perhaps even worse is that we are seeing more reactance in 2018 as well. People aren’t just skipping through questions more quickly in 2018 and lying more often. They are also significantly more likely — in fact, 143 percent more likely — to directly attack the survey authors or otherwise rebel against the survey. For example, when asked to describe themselves, they are more likely to say things like, “mad that I have to fill this out” or “someone who hates nosy strangers.” And when asked to describe their most important personal values, they are more likely to say things like, “not doing surveys” or “none of your business.” It should be obvious why these kinds of responses are problematic and not valuable for us as marketers.
Take these trends seriously. The figures as they stand are not great, but they can get worse — to the point of survey data becoming completely worthless. Be 100 percent transparent with people about the information being asked for and how it will be used. Because right now, people aren’t so sure they want to tell us anything. If people don’t want to take surveys anymore, a unique channel of learning will become forever closed.
Hillary Haley is svp and executive director of behavioral science at RPA.