Online Tool Store
· 4 min read
How to Write a Listener Survey People Finish
Heshan Fernando
Co-founder & COO
A twenty-question survey gets forty responses from an audience of several thousand. Those forty people are the most engaged listeners you have, and they are not representative of the audience you were trying to learn about.
Survey length determines who answers, and who answers determines what you learn.
Length selects your respondents
Completion rates fall as surveys get longer. That is unsurprising and it is not the main problem.
The main problem is that the people who persist through a long survey differ systematically from those who abandon it. They are more engaged, more invested, and more likely to be your existing enthusiasts rather than your marginal listeners.
If the question is “should I change the format”, the people most likely to say no are exactly the ones most likely to finish a twenty-question survey. The result is biased in a direction you cannot correct for afterwards.
Under eight questions keeps completion high enough that the respondents look more like the audience.
One goal per survey
A survey trying to answer content direction, sponsorship demographics and format feedback in one pass does none of them well.
Each goal wants different questions, and combining them makes the survey longer, which makes the sample worse, which makes all three answers less reliable.
Pick one:
Content direction — what listeners want more and less of. Format feedback — length, structure, frequency. Sponsorship data — demographics and purchasing relevant to advertisers. Discovery — how people found the show.
Run another one in three months for the next question.
Leading questions produce data you will act on
The most damaging survey error, because the results look like evidence.
“What do you like most about the new format?” presumes the listener likes it. Someone who does not either picks the least bad option or abandons, and the results say the new format is popular.
“What do you think of the new format?” does not presume, and it gets an answer you can use.
Other patterns to watch:
Loaded words. “Do you agree the episodes are too long?” against “How do you find the episode length?”
Double-barrelled questions. “Is the show informative and entertaining?” cannot be answered by someone who thinks it is one but not the other.
Unbalanced scales. Options running from “good” to “excellent” with nothing below.
| Question | Problem |
|---|---|
| “What do you like most about X?” | Presumes liking |
| “Do you agree X is too long?” | Loaded |
| “Is it informative and entertaining?” | Double-barrelled |
| “How do you find X?” | Neutral |
Demographics last
Age, location and occupation are the least engaging questions in any survey and the most likely to cause abandonment.
Asked first, they cost you responses to everything else. Asked last, someone who abandons has already given you the answers that matter.
They are also the questions people are most likely to skip or falsify, which is another argument for not spending early goodwill on them.
If they are the point of the survey — for a sponsorship deck — keep the survey to almost nothing else.
Open text costs completions and earns insight
A trade worth making deliberately rather than by default.
Closed questions — scales, multiple choice — are fast to answer and fast to analyse. They also only return answers you thought to ask about.
Open text returns things you did not anticipate, which is frequently the most valuable output of a survey. It also takes longer to answer, increases abandonment, and requires reading rather than counting.
The workable balance is one or two open questions, placed near the end, phrased specifically. “What would you change?” gets vague answers; “What is one thing you skip when it comes up?” gets usable ones.
Making them optional preserves completions from people who will not write anything anyway.
Common mistakes to avoid
- Twenty questions, which selects for your most engaged listeners.
- Trying to answer several different questions in one survey.
- Leading questions that produce the answer you expected.
- Demographics at the start.
- Running a survey with no decision attached to the result.
How to do it with Podcast Listener Survey Builder
The Podcast Listener Survey Builder flags leading questions.
- Pick one goal for the survey.
- Keep it under eight questions.
- Check the flagged questions for presumption and loaded wording.
- Put demographics at the end, if you need them at all.
Other creator tools are in the tools directory.
Frequently asked questions
How long should a listener survey be?
Under eight questions. Longer surveys are completed disproportionately by your most engaged listeners, which biases the result in a direction you cannot correct for.
What makes a question leading?
Presuming the answer. Asking what someone likes about a new format assumes they like it. Asking what they think of it does not, and the difference shows in the results.
Why put demographics last?
They are the least engaging questions and the most likely to cause abandonment. Asked at the end, anyone who drops out has already answered what matters.
Final thought
Decide what you will do with each answer before writing the question. Any question where the answer changes nothing is a question costing you a completed response.