Simplifying Surveys

Data is touted as the new currency of the twenty first century. Designing a good questionnaire can be the difference between solid reliable data and a bunch of meaningless numbers. Governance in the 21st century is going to be increasingly data driven and policies built on good data have the potential to revolutionize governance and policymaking.

Here, based on our experience, we take a look at some key guidelines that could be worth remembering when designing a questionnaire.

Length of Questionnaire

The length of a survey directly affect response rates, survey costs, and data quality. Longer questionnaires result in higher costs and may lead to poor quality of response. However, research shows that below a specific threshold, questionnaire length bears little relationship to response rate or data quality and has only a minor impact on survey costs. In our experience, any survey lasting more than twenty minutes tends to elicit poor responses and frustration from the respondents.

Asking the right questions

Before going around building a questionnaire one must clearly identify what the survey seeks to find. This can help designing the questions and choosing the respondents. The questions need to be precise and short with minimal ambiguity. Leading questions must be avoided at all costs with a neutral tone maintained throughout. Deciding whether the questions need to be open or closed ended, quantitative or qualitative depends on the research question and the kind of data that is needed.

Choosing the right responses

Framing the responses to a question is as important as framing the question itself. Responses to open ended questions can be accurate and qualitative but can be very hard to code. Close ended questions need to provide the respondent with enough flexibility to elicit accurate answers. Instead of going for a binary response (Yes/No), displaying responses as a spectrum (Likert scale) or a matrix often helps get more accurate data.

Ordering 

A questionnaire must flow logically with questions of the same domain clubbed together. A survey should begin with simple questions that disarm a respondent and make him feel comfortable. Any controversial or sensitive questions must be left towards the end. Within similar set of questions, the order must flow from the general to the specific.

Customization

Any questionnaire must be customized for the local conditions and customs. Surveys that do not keep local sensitivities in mind can deliver poor data at the end. One way to ensure the questionnaire stands the test of reliability, that it can produce the same result if administered again, is to perform pilot testing. The testing, if performed by the framers themselves would help them understand where the questions have been framed incorrectly or how the sequence of the questions can be improved.

Designing a questionnaire for a survey is a complex yet crucial activity. It is as much an art as a science it is made to be. Hope these quick tips help you simplify the process and design an effective survey questionnaire in the future.

Nixon PJ is a Data Associate at Ank Aha.