Summary

Why should you care about double-barrelled questions? They confuse people and make you look like you are either being manipulative or just plain incompetent. They waste EVERYONE’s time, including yours.

Video Transcript

Why Should You Care About Double-Barrelled Questions?

Karyn: Today we are talking about double-barrelled questions and how horrible they are and that sort of thing. So just to get started and so everybody knows who might be watching. Maggie, why should we care about double-barrelled questions?

Maggie: You should care about double-barrelled questions for a number of reasons, but most importantly, double-barrelled questions can confuse respondents and when respondents are confused, they are questioning your authority and legitimacy and it’s going to weaken your position and not lead to any useful data or valuable insights.

Karyn: That’s the best answer I’ve ever heard to that question. That is why it matters. When people get confused, they quit, they stop. If you don’t understand the question, you skip it or just say I’m done. Thank you. Close window.

Maggie: Yes. And they totally don’t trust you because they’re like, wait, this person’s an expert in this and they don’t know how to ask me a question about it? No.

Introductions

Karyn: Alright, well just so you know who we are. My name is Karyn Kelbaugh and I do customer research, and evaluation for nonprofits and you can find me at heykaryn.com.

Maggie: I am Maggie Hodge Kwan and I do evaluation and research work for nonprofits and philanthropic foundations. And you can find me at creativeclarity.ca.

What is a Double-Barrelled Question?

Karyn: So today we’re going to try to keep it shortish, and get into our series of nerding out about questions. And today, as we said, we are talking about double-barrelled questions. And so let’s just talk about defining what is a double-barrelled question. Like to me, it is when you are asking more than one question but you’re trying to jam it into one question. Like what’s your favorite color? And do you like strawberries? A very simple version, but they can be a lot sneakier to where you don’t really think it’s a double-barrelled question. You had a really good example, like when you’re trying to connect two ideas together that don’t necessarily go together.

Examples

Maggie: Or even when they do. And the example that I can think of here is from a quality of life survey put out with the best of intentions, but asking many questions within questions. And one of the things that stood out to me – it said something like, “My municipality encourages and supports small business”. And off the top you think, oh, that’s interesting. This is gaging my perception around whether or not the place that I live support business, but you look at those two separate concepts and encouraging small business is one thing and it probably has to do with it with the attitude. Supporting small businesses is another thing and that could be with bylaws, policies, incentives, all those kinds of things. So as a respondent, I feel like I’m in a quandary because I might have two different opinions. I might think yes, they’re very vocally encouraging of small business, but no, there aren’t really a lot of actions that support small business.

Karyn: Yeah. It’s like if you can give two different answers to a question, it might be a double-barrelled question.

When you are saying “Yes, And” or “Yes, but”, that means that it’s probably a double-barrelled
question. So that’s kind of a good way to, to think about it, to recognize it. If you could break it down into separate questions, it’s probably a double-barrelled question.

I’m trying to think of another good example. They’re not just questions. You had one that was a statement. So it’s do you agree or disagree with this? And I think there was another example where we were talking earlier, the climate one. Yeah. They, sounded like they were trying to, and I’m going to be nerdy here, so just get over it, put causation into the statement. Like this happened because of this or something like that. Where they’re just trying to bake as much as possible into the question or statement. Do you remember that statement?

Maggie: Yes. And the statement was something like, “climate minded citizens support active transportation”. So, you know, walking and cycling routes and public. Sometimes public transportation can be related to that, but very much seeding the idea that people who care about climate change and people who want to mitigate climate change would automatically support things like sidewalks and cycling infrastructure. So just a lot going on. There are many issues.

Karyn: That one has a whole lot of problems. Yes, let’s fill a sentence full of jargon and see if anybody can tell what we’re talking about. And also the fact that you get to the end of the question, and you think, so what? We could go all day (haha) which is why where we’re making more than one video. But the idea of trying to shove two ideas into a statement and then asking do you agree or disagree with this? And you’re thinking “well, sort of”. If you can’t give a clear answer. If people can’t have an answer pop into their head immediately, then you’re confusing them and they’re going to leave.

Maggie: Yes. So, because that’s the kind of, I mean, Karyn, I think you know this probably better than anyone, but in a survey specifically, you want the experience to be as easy as possible for people. You want to use plain language so that people understand what you’re asking and you want them to be able to respond without having to take 10 minutes to think about exactly what’s being asked. So those darn double-barrelled questions really throw a wrench into that and give people pause and if you’re anything like me, make me kind of paranoid that they’re going to use the data in a weird way or that they’ll just have to throw it out and then they’re wasting my time asking me about it.

Not Always Better Together

Karyn: Yeah. I could just picture this scenario where somebody is tempted to do it and they don’t realize they’re doing it when they’re thinking, well, you know, I wrote this survey and it’s got 15 questions. I feel like that’s too long, so I’m going to make it 10. But I still want 15 questions worth of answers. So instead of actually narrowing down their questions based on the best questions, they just combine them thinking it will work.

And there’s a difference between asking a question and then having an immediate follow-up question. Literally a separate question, but it’s together. That’s one thing. Like you can kind of combine that to where I’m clearly asking you two questions, but I put it on one line and I made it a follow up question. That’s different than I’m going to ask what is your opinion about this and your opinion about this? Like I can’t just shove them together, but it can be really tempting when you’re trying to make it shorter to just say, well we’ll just jam them together and it’ll be fine. You’re better off just having a longer survey. I mean, you’re better off having a longer survey that asks clear individual questions than you are trying to jam them together just for brevity.

Maggie: Absolutely. And so, you know, hypothetically a grocery store wants to know what its customers are buying and let’s just ignore for now the fact that of course, they would know because they have all the sales data, but you know, say that they’re thinking, okay, we only have five minutes of our customer’s time. We need to ask you know, we need to cram in as much as we can. And so a customer gets a question that says “I buy apples and oranges” and the responses are always, frequently, sometimes, or never. There’s really no winning there. Right? And that would be a case where you would, it would make much more sense to break them into two questions and figure out how often people are buying apples and how often people are buying oranges by asking about those things separately.

Karyn: Or we could change the whole context of the question. Like how often do you buy fruit?

Maggie: Love it.

The Why Matters

Karyn: You’re going to hear us say this over and over again in every single video so just start putting it in your head. Why you’re asking the question matters. What are you trying to find out? What is the purpose of the question so that you can come back to that every single time. Like, why do they care how many apples and oranges they’re getting? And kind of just getting to the point of that. That can always bring you back when you feel like your questions might be getting a little out of control.

Is there anything else on double-barrelled questions that we should cover? Do you think that pretty much hits it?

Check Your Beliefs

Maggie: There’s just one more thing that I wanted to add, and that is that double-barrelled questions, as you catch yourself writing them, are a really good opportunity to check your own attitudes and biases that you might be bringing into a piece of research. So as you’re writing a survey, if you are that municipality and you’re writing, “my municipality supports and encourages small businesses”, that might be an opportunity to figure out, am I just trying to find out as much information as possible within a small survey or as a survey writer, is my own belief sort of bleeding in here? And how could I take a look at this question and reword it so that it’s asking one thing in an objective manner that isn’t biased by the survey writers experience or attitudes?

Karyn: Which totally touches on the topic that we’re going to talk about in another video, which is Leading Questions, which I’m just. There’s so much to talk about with leading questions, but for now, thank you.

That was awesome. Thank you for joining me. So that’s it for now for double-barrelled questions, but if you have any follow up questions for us, wherever you are watching this video, just send us a comment and we would be happy to get back to you on that. And if you found the content valuable, we accept exposure for payment so you can click like or share it with anyone that you think would find it valuable. And if you want to follow along with our question series, you can subscribe to the Youtube Channel or whatever page you are watching this on. There will be quite a few more. We’re going to get into other stuff like leading questions. And Maggie had a fabulous one which we’re totally going to do, which is developing the shortest most effective survey which totally ties into today. And other fun stuff like when it’s better to use an open-ended question versus a multiple choice question and all kinds of other survey format stuff. So thank you very much and we’ll talk to you later.