Creating surveys which are meaningful and ask the right questions of the right people is not always an easy task. Our team regularly receive requests from colleagues asking for advice around survey design, and we thought it would be helpful to share some of our top tips with you.
Wait…is a survey the right method?
Before we get on to top tips, it’s first worth noting that there are many different ways you can engage with people, and surveys aren’t always the best option.
Surveys provide a simple way of gathering large amounts of information in a short space of time, for a relatively low cost. However if the subject matter is sensitive, if you’re looking for deeper insight into people’s experiences or to understand the reasons behind opinions and ideas, face-to-face research methods might be more suitable.
Define the purpose
Let’s say you’ve decided a survey is the best method. Now to think about why you are doing it – what is the purpose of engaging people and what insight do you need? What are your key research objectives and what decisions will the insight influence?
Start off by mapping out what insight you need, and then develop questions based around this. This might sound simple, but it can be all too easy to throw in extra questions which might be ‘nice to know’, but don’t actually provide insight which inform your key aims.
When designing the survey, each time you come up with a new question check back with your list of objectives and see whether it helps to inform these. If it doesn’t, think about whether you really need it.
Know your audience
Before you start, it’s essential to know who you want to gather views from. The more you know your target audience, the easier it will be to tailor your questions to be appropriate and relevant for your respondents. This is particularly important when gathering views from for example children and young people, or those with learning disabilities or communication difficulties.
Regardless of who the survey is for, keep the language simple and avoid jargon. As well as acronyms, this includes rethinking some favourite public sector terms which are part of our everyday language, but might not mean an awful lot to the average member of the public. I’m talking words like ‘community assets’, ‘co-production’, ‘stakeholders’, ‘commissioning’…you get the idea! It’s not about ‘dumbing it down’, but taking some time to think about how to phrase your questions so that they can be best understood by your audience.
Knowing your audience will also help you to promote the survey in places which will reach the right people. For example if you want to reach groups who don’t typically engage with council services, you might need to think outside the box a bit and consider sharing it in places where those groups tend to be – both in the community and online.
Creating the questions
Here are a few general principles to help guide you in developing survey questions.
Keep it short and sweet
This isn’t always realistic depending on the purpose, but it’s best to keep surveys as concise as possible, and don’t ask more questions than you need to.
Go easy on the open-ended questions
A page full of free-text boxes looks scary, and will put respondents off. Use multiple choice questions where appropriate – this will make it easier for the person filling it in, it helps to quantify responses, and also allows you to easily identify trends and comparisons across respondents.
A combination of open and closed questions can work well – use closed for direct feedback on specific topics, and open questions to understand the ‘why’ behind these responses, or for further exploration around areas of unknown.
You wouldn’t use a leading question, would you?
Leading or biased questions encourage people to respond in a particular way, and may end up producing responses which aren’t a true reflection of how people really feel. Try to use neutral language that doesn’t include words which are emotive or have positive or negative connotations, e.g.:
Leading: Do you agree that our customer service was excellent?
Not leading: How would you rate our customer service on a scale from 1-10?
Think about flow
The ordering of questions can make a big difference to the feel of a survey and the experience a respondent has when completing it.
The order which works best will depend on both the topic and the length of the survey. For example if a survey is longer in length, you might want to put the most important questions at the beginning so that they are answered first (in case the respondent loses interest half-way through – we’re all guilty of this!)
However if the subject matter is slightly more delicate, you might want a couple of ‘warm up’ questions to ease respondents into it first. Set your questions out in a logical way that make sense to you, and bear in mind it’s always helpful to group questions on similar themes together.
Remember that someone has to analyse the responses
Surveys can produce a whole lot of data, so it’s good to think about the end point and the fact that someone will have to make sense of all the responses. Keep this in mind when creating your survey – with open questions, try to be specific so that respondents’ answers don’t deviate off topic. This will make analysis and theming of comments much easier.
Take a look at our new consultation platform!
Surveys of course need to be hosted somewhere, and we’ve just launched a new consultation platform which will hold all of our future surveys and consultations. We’ll be blogging more about this soon!
In the meantime, we’d love to hear about your experiences of running surveys – what challenges have you faced and how have you overcome them?
Leave a comment