For many researchers online survey tools such as survey monkey have really changed their lives. Online surveys, not only it make designing a questionnaire easy but it also provides access to your target population. Those days of hard fieldwork knocking on doors or stopping people on their way to the supermarket are gone. All you have to do now is wait in front of your computer for your respondents’ answers to come to you.
But there are a few things you should be aware of if you are relatively new to survey
monkey or similar tools. It is very important to get the advice from an expert, ideally a statistician, at the moment of designing your survey questions using online surveys. You need to first decide what type of answers you want for each of your questions. First of all, is your question going to be measured as a continuous or categorical variable? For instance, you could have age reported as the exact years of age of the respondent or by offering age groups from which the respondent will choose (e.g. below 20, 21 to 25, 26 to 30 and so on).
When you define a question as multiple choice you need to ask yourself weather you will allow for multiple answers or a single one. This is probably one of the most common problems I have encountered with clients who come to me with a database extracted from survey monkey. Categorical variables by definition should only accept a single response. The reason is that respondents have to be classified uniquely since categories are mutually exclusive. For example, to a question about gender a respondent cannot choose male and female. Or to a yearly salary question s/he cannot choose categories 30k-40k and 50k-60k at the same time.
On the other hand, there may be questions that are multiple answers, for example, “choose the 3 top reasons why you would decide to change your internet provider” or “choose 3 of your favorite cities”. These type of questions are in fact measured by a set if binary variables that take 1 if that answer has been selected and 0 otherwise.
The problem occurs when the researcher defines a categorical variable as a multiple answers question. Two are the main problems that derive from this. The first one, it will create as many columns in the database as categories are defined for that variable instead of one single column. Although having dummies for your categorical variables can be useful for some future analysis you still want to have the original variable with all its categories so you can tabulate it and crosstabulate it with other variables. Therefore, you will need to go across all your questions one by one and collapse the many columns created into one.
Not only you will face this time consuming issue, but what can be more problematic, you will allow for multiple responses. Give the choice to respondents to choose more than one answer even if it makes no sense and, trust me, some of them will do it. So you will need to decide what to do with those individuals who have selected more than one answer. Do you drop them from your analysis or do you randomly disregard the extra answers? These are all decisions that you could have avoided if you would have defined your categorical variables to allow only one answer.
Another issue related with working with survey monkey or similar online surveys is that the basic version will only allow you to export your data to excel and all your categorical variables will be exported with string and not numeric format.
Having said all this I still believe that survey monkey and other similar online tools are here to make your life easier but be careful and use it wisely.
