Survey Research Services

DataForce Survey Research Services Information, Tips, and How-tos

Effects of Badly Worded Survey Questions  

In questionnaire design text, it is both important that question-wording should express what is being measured, and that appropriate response options are provided. However, there are times the latter is overlooked, and a mismatch occurs between the question wording and response options. So, the big question is, how detrimental are these mismatches? Below, we discuss the effects of badly worded survey questions (how mismatches between survey questions and response options affect a study). We hope this information will help you weigh whether it is more critical for questionnaires to firmly match survey wording, or give more significance to more straightforward and less wordy options.

Common Sample of Mismatched Questions

effects of badly worded survey questions

effects of badly worded survey questions

Effect of Mismatched Questions

  • Higher Item Nonresponse

Mismatched questions will result in higher item nonresponse because of the difficulty in responding due to additional cognitive processing. In responding to survey questions, respondents must identify the question, understand it, recover relevant information, create a judgment, and then report (Jenkins and Dillman 1997; Tourangeau, Rips, and Rasinski 2000). Consequently, there is a probability some respondents will skip items, or refuse to answer when they are troubled to exert additional intellectual effort. In an experiment by Dillman et al. (2014), they found that the mismatched version caused greater item nonresponse, and differences in response distributions, compared to the matched version. The study varied the question stems paired with forced-choice response options to be matched or mismatched with check-all question stems.

  • Longer Response Time

Mismatched versions will take longer to finish compared to matched versions because of the added time-consuming comprehension and mapping complications. Additionally, the mismatch may only be identified by the respondents when they are mapping their answer to the response options and determine they are not appropriate. In this case, they have to exert additional effort in finding an adequate answer in the response option to replace their answer. Van der Zouwen and Dijkstra (2002) determined that mismatches or poorly designed response options (inadequate range of response alternatives) were greatly connected with swaying from the classic interviewer/respondent interaction in a question/answer sequence. An example of a classic interaction is an interviewer asking exactly worded questions to respondents, and receiving an adequate answer from them. Mismatch tends to result in respondents asking the interviewer to clarify their question further, resulting in longer survey completion time.  

Confirmation by a Recent Study of the University of Nebraska-Lincoln: “The Effects of Mismatches Between Survey Question Stems and Response Options on Data Quality and Responses”

The experiment divided the respondents into two groups. The first group received a survey version with all answer options and response options matched, and the 2nd group a slightly mismatched version. Their theory was that mismatched questions would result in higher item nonresponse and longer response time.

Example Matched and Mismatched Survey Questions from the Study

Results of the Study

The item nonresponse rate is higher in the mismatched than in the matched version. The probability of an item being unanswered in the mismatched version compared to the matched version is 1.6 times. 

Response time was longer in the mismatched than in the matched version. During the survey conducted via telephone interview, the researchers found that the interviewees took more time to complete the mismatched version, compared to the correctly matched one. 

The effect of the mismatch is not greater for those with lower cognitive abilities based on age or education. 

Summary

The study confirmed that mismatches weaken data quality in both mail and telephone surveys. Although, it may be less detrimental for the mail survey because it is self-administered, and respondents can see both question stem and response options. Researchers should design their questionnaires carefully, taking into consideration the proper relationship of the question stem and response option. Moreover, in analyzing survey results, researchers should review both stem questions and response options to determine if a mismatch could have made an impact on the results. 

For more information on Survey Design, survey mailing services, and data capture services, quantitative data collection services in general, contact us!

By |2020-06-27T08:58:34+00:00June 20th, 2020|Survey Research Services|0 Comments

What Survey Mode is Best for My Project

While there are many tools available for data collection, surveying is one of the most commonly used. Surveying is essentially a research method used to gather data from a sample of people to generalize results to a larger population and gain insights on various topics. Questionnaires are used in asking people for information in a survey. As compared to other data collection methods, such as direct observation and experimentation, surveys yield a broader range of information. The most common survey types by distribution are online, telephone, face-to-face, and mail survey.

 

Types of Survey

 

1. Online Survey

Different Types of Survey

 

Technology has enabled online surveys to become the most popular and cost-effective type of survey. The questionnaire can be completed with a smartphone, tablet, or computer so long as the respondent has access to the internet. 

a. Advantages: 

  • Reach of the survey has increased to wherever there is internet access.
  • No limit to the type of questions that can be asked.
  • Data collection and data analysis is now structured and easy to manage.
  • Ideal for short, simple surveys.
  • Quick results.

b. Disadvantages:

  • We receive requests to take online surveys frequently, whether you make a purchase from Amazon, pick up lunch at McDonald’s or take a class; everyone wants us to take their online survey. This saturation has contributed to survey fatigue, and people are only doing online surveys if they are unhappy or related to something meaningful to them.
  • Inboxes are inundated nowadays; we are busy looking for the things that need our attention and delete or skip the other ‘stuff.’
  • People are hesitant to click on links that come from people they don’t know, they don’t want their information stolen or their computer hacked. 

 

 

2. Telephone Survey

 

The medium used to contact respondents is the telephone. An interviewer follows a script in asking a specific set of questions to the respondents, and a data entry software is used to record the respondent’s answers. 

a. Advantages:

  • Relatively cheaper and less time consuming than face to face surveys. 
  • Extensive geographic access since most people in the United States have a telephone or cellphone. 
  • Easy access to in-house or online phone directories. Phone numbers can easily be purchased from sample companies. 
  • Time effective since interviewers can just keep calling numbers until they reach their quota.
  • Skilled interviewers can elicit longer or more complete answers. Interviewers can also ask for clarifications of unclear responses.  

b. Disadvantages:

  • Hard to make a connection with people since interviewers can’t see the person’s reaction. 
  • Intrusive, since most of the time, telephone surveys are done without notice. The interviewer might be interrupting the respondent’s plan for the day. The researcher must carefully consider the time and length of the call.
  • Interviewers may be perceived as telemarketers and, consequently, turn-off respondents. 
  • Regulations must be followed to avoid significant fines.

 

 

3. Face-to-Face Interview

Different Types of Survey

 

Face-to-Face surveys are one of the oldest and most widely used survey types. The researcher typically interviews in the home, office, hangout place, etc. of the target respondent. This is by far the most personal approach and best used if you are looking to raise trust and cooperation from respondents. Interviewers must be trained well, including on how to read non-verbal cues to direct the interview better.

a. Advantages  

  • Can capture verbal and non-verbal cues. The interviewer can gauge if body language and facial expressions match the participant’s answer.
  • The interviewer can make sure that the participant is committed and encouraged to finish the survey.
  • The interviewer can provide assistance in case the participant is confused about any part of the survey or question.
  • The interviewer can take advantage of the five senses. Aside from audio and visual stimuli, the researcher can also let respondents touch, taste, and smell materials to support the interview.

b. Disadvantages

  • A face-to-face survey can take longer. Interviews can last for days or weeks, depending on the number of respondents needed and their availability.
  • Considerably more expensive than paper, online, and telephone. Training, travel, and material are some of the principal costs. 
  • The quality of data depends on the skill of interviewers. 
  • It requires more effort to plan and manage.

 

 

4. Mail Survey 

Different Types of Survey

 

For reasons of cost and ease of implementation, mail surveys are more frequently used for social research than are either telephone or face-to-face interviews, according to Don Dillman.  Before online surveys, 69% of surveys were conducted solely by mail and another 11% were a combination of mail and some other mode. 

a. Advantages

  • It can be used when the respondent’s internet access or knowledge is limited.
  • Less expensive than Face-to-Face or Telephone surveys.
  • Allows respondents to complete the survey at their convenience. 
  • A hard copy serves as a reminder to finish the survey.
  • Research shows that respondents give more honest answers when compared to other modes.
  • Respondents trust mail surveys more than online surveys since we are told not to click on links from people/organizations that we don’t know. 
  • You have less competition with someone’s mailbox than you do with their inbox.
  • Best for capturing sensitive information or long, complex surveys.

b. Disadvantages

  • Respondents may not follow directions or only answer certain questions, leaving an incomplete response.
  • It takes more time than online surveys to collect the data.
  • If your study requires alternate question sets or alternating question order, paper surveys may be too costly to support this requirement.

Check out our earlier post on Why Mail Surveys are Thriving in the Digital Age.

 

How to Select the Best Survey Type for Your Research

 

1. Consider Population and Sampling

Define the characteristics of the target respondents that belong to your population before you choose a survey method. Determine geographics, language, communication, literacy, and other issues that might arise. For example, if your target respondents are older people, you may choose a mail survey; however, if the target population is younger or more tech-savvy, an online survey might be more appealing.  

2. Determine Question Types 

In selecting the right survey method, the questions that the respondents need to answer should be considered. Paper or mail surveys can be ideal for mostly closed-ended questions, while a survey with plenty of open-ended questions that could require a follow-up may require a face-to-face or telephone interview.  

3. Check your budget

You have to justify the cost of the type of survey you will choose. You may want to do a face-to-face interview, but the costs compared to a mail survey may not validate the benefit of pursuing face-to-face interviews. 

4. Establish your timeline

Some survey methods take longer to complete than others. If you are in a rush, then an online survey is your best bet. However, if time is not a significant factor, then you can do a mail survey.

5. Check Access to Facilities and Resources

Do you have the facilities, equipment, and human resources needed for your survey to run smoothly? If you plan to do a mail survey, you need access to printers, human resources or equipment for stuffing envelopes, data processing software, warehousing, and so on. For telephone surveys, you need well-trained interviewers, phone equipment, CATI software, etc.  

It’s important to know which survey type to use and when to use it. Once you’re familiar with the different survey types, you’ll be able to focus on what you need to make your survey distribution as smooth as it can get, getting you far better results than ever before. For more information on data collection techniques or any aspect of mail survey management, contact us today! We provide outstanding quantitative data collection services and paper scanning services!

By |2020-05-19T21:44:22+00:00May 19th, 2020|Survey Research Services|0 Comments

What is a Likert Scale and How to Create One

Are you interested in finding out what you can use to measure questions that are neither agreed nor disagreed? A Likert scale can help you measure attitudes and opinions with a greater degree of nuance than simple binary questions, which offer only two answer options. Please read our blog post to learn what is a Likert scale and how to create one for your next survey.

1. What is a Likert scale? 

The Likert scale is one of the most popular rating scales developed to measure one’s attitudes or opinions. Fixed choice response formats are used to determine how people feel about the topic, products, services, or experience. The scale deems that the strength/intensity of the experience is linear. These linear scales measure points of agreement/disagreement. People are given five to seven choices, or even nine balanced responses, that often come with a neutral point. 

 

2. Common Likert Scale Question

A Likert scale does not have a fixed number of leveled items. Many researchers use five levels, but some also use 4, 7, 9, and even 10-leveled items. Since adding more levels produces diverse valuations, a 5 or 7 level scale is most often ideal for avoiding extreme options by obtaining just a bit of variation.

Below are some examples of Likert scale questions and answers:

a. Agreement

The employee training provided the knowledge I need to do my work efficiently.

  • Strongly Disagree
  • Disagree
  • Undecided
  • Agree
  • Strongly Agree

b. Satisfaction

How satisfied are you with our customer support?

  • Highly Dissatisfied
  • Dissatisfied
  • Neutral
  • Satisfied
  • Highly Satisfied

c. Frequency

How often do you visit our store?

  • Very Frequently
  • Frequently
  • Occasionally
  • Rarely
  • Never

 

3. When to Use Likert Scales

What is a Likert Scale

Likert scale is useful in measuring the general feeling or opinion of a particular topic, product, services or experience, and collecting additional data on the factors that contribute to those feelings or opinions. However, a Likert scale should only be used when the question items are related to each other and can be presented in a degree-scale form. Since respondents are not limited to a yes/no answer, a Likert scale allows researchers to obtain quantitative data that can be easily analyzed. Nevertheless, a Likert scale may be compromised because of “social desirability”. Social desirability is the bias exhibited by people to present themselves in a positive light in the community. For example, in taboo questions involving sex, illegal drugs, or racism, respondents may heighten “good behavior” or depress “bad” or undesirable behavior of their responses. One way to reduce social desirability bias is by allowing anonymity on self-administered surveys. A study by Paulhus (1984) found that when respondents have to put their name, address, and telephone number on the survey, results show more positive personality characteristics than an anonymous survey.  

 

4. How to create a Likert Scale

Establish the footing of your survey questions and response scale by first deciding what you want to measure. It is best to use a Likert scale when several factors are influencing the way your respondents feel about something. For instance, you want to measure patient satisfaction. Many factors affect patient satisfaction, including affordability, general behavior of doctors, amenities, and administrative procedures. The respondents’ opinions, attitudes, feelings, or experience must be measurable in a scale form. Moreover, make sure that there are two well-defined extremes for the response.  

For Example:

What is a Likert Scale

 

Recommendation

  • A Likert scale should have the same number of positive and negative responses.
  • Stay odd. Provide your respondents with a neutral option. 
  • Use the appropriate description to label response. When you just use numbers, people may obscure which end is affirmative and which is undesirable.
  • Make sure your survey questions are specific
  • Use terms that your target audience understands
  • Avoid bias questions
  • Avoid long and complicated questions
  • Avoid double-barreled questions

Check out our post on How to Write Great Survey Questions

You’ve most likely encountered Likert scale questionnaires without even knowing it. Likert scale questions are valuable for assessing people’s opinions on a specific topic when undertaking in-depth research. 

For more information on data collection techniques or any aspect of mail survey management, contact us today! We provide outstanding quantitative data collection services and paper scanning services!

 

Get more information on – survey research services incentive fulfillment services or survey mailing services

What’s the Difference Between Qualitative Research and Quantitative Research?

To understand a problem, large or small scale, both high quality and relevant data are needed. If you’re a researcher, you’ll have to collect this data using Qualitative Research or Quantitative Research methods. Without a solid understanding of the difference between qualitative and quantitative research, you risk using a less suitable method of data collection, impacting your results, and ultimately your study. 

In this post, we’ll discuss all the things you need to know about qualitative and quantitative research, their differences, and how to best use them for the success of your project. 

Content 

  • Definition of Qualitative and Quantitative Research
  • Table of Comparison of Qualitative and Quantitative Research
  • Key Differences and Example

 

Definition of Qualitative and Quantitative Research

Qualitative Research uses non-numerical measures to understand fundamental behavior, reasons, and motivations. Unstructured or semi-structured methods are utilized to gather insight into questions. This stage is often considered as the exploratory stage of research where freedom in response is implemented to gain an initial grasp of a problem.

Quantitative Research

  • For Example: An open dialogue with participants to find out why customer satisfaction ratings are low. Data from qualitative research can later be used to generate quantitative data to develop ideas or solutions. This is done by tallying the frequency of the qualitative result and determining the factors which are most relevant.

 

Quantitative Research is used to assess a subject by utilizing data that can be measured and then interpreted using statistical evaluations. Correspondingly, closed-ended questions are applied, and the target audience is given a set of options to use for responding. Quantitative data provides numerical facts!

 

Quantitative Research

  • For Example: The percent of people that find fulfillment in their job, or think their trainer is effective. The analysis of opinions, attitudes, and behaviors are simplified by using numerical data to draw conclusions, test a hypothesis, and discover trends in research. 

 

Table of Comparison of Qualitative and Quantitative Research

Quantitative Research

Key Differences and Example

The key differences between Quantitative and Qualitative Research are the purpose and methodology.

  • For Example: A School Board wants to find out how many of their students feel their teachers care about them. They use an in-house school survey composed of closed-ended questions to provide a numerical measurement. Let’s say the school finds out that 800 out of 1200 of the students don’t feel their teachers care about them. This is quantitative research. The purpose of quantitative research is to provide a concrete numerical measure by using the closed-ended methodology in order for the response of the students to be measured on a common scale. Furthermore, the school board decides to dig deeper into this issue. Consequently, they create a focus group discussion composed of 5 students from each class. The goal is to understand the key causes through open dialogue. The result of this research will be the opinions and statements of the chosen students. This is qualitative research. The purpose of qualitative research is to provide a deeper insight by looking at the human perspective, hence, using an open dialogue for respondents to sufficiently express themselves. 

 

Do You Need a Partner in Your Research?

Partner with DataForce Survey and Study Management

Our expertise in high-volume, paper-based, and multimodal data collection projects encompass the entire data collection supply chain. We can provide you with end-to-end data collection services, or participate in any part of the supply chain where you need assistance – from project scoping to form design to printing, mailing, fulfillment, data collection, data delivery, result reporting, and analytics. Contact us now!

 

For more information on Survey Mailing services and Data Collection Servicessurvey fulfillment services  Contact us Now!

By |2020-05-28T07:03:20+00:00May 11th, 2020|Survey Research Services, Uncategorized|0 Comments

3 Tips to Streamline Your Survey Return Schedule

Effective mail surveys are typically planned and executed like a well-choreographed dance routine that must have all dancers hitting their mark at the right place and the right time. Every step in the sequence – from printing and mailing to fulfillment and data collection services – must be optimized for a streamlined performance.

One step that requires particular attention is the timing and management of survey returns. Survey return management depends on factors outside your control, including the timing of respondents’ completing the surveys and the postal service delivering the returns. This relative blind spot also creates challenges in staffing. You can imagine the wasted cost in staffing a team to process more than 10,000 returns while facing unexpected survey return delays.

As a leading provider of mail surveys, we help researchers plan for and anticipate potential risks and/or delays before they happen. Here is what we found are the best ways to streamline your survey return schedule:

 

1. Set an Appropriate Response Time Window

Survey response time is driven by several factors:

  • Interest Level A high-interest topic and a short survey could see returns in as little as a few days, whereas others can take as many as one to two weeks. Being mindful of this will help you set reasonable timeline expectations. 
  • In-home Date Depending on your mail delivery method (standard vs. express), your respondents may receive the survey in-home in as little as a few days or as long as 1 week or more. Consider in-home receipt as part of your timeline window.
  • Return By Date A clearly communicated “complete/return by” date should be printed on the survey materials so that respondents have a deadline.
  • Holidays – Holiday mail takes longer to be both delivered and returned. Respondents also tend to put off completing their surveys during holidays as distractions abound. If you must send out your surveys during a holiday, allow extra time for returns. We typically recommend a 3-4 week return window for most surveys, and 5-6 weeks during holidays, to ensure participants have enough time to consider and complete their surveys.

 

Survey Return Schedule

Survey Return Schedule

(3 mailings, $2 pre-incentive, $40 promised incentive)

 

2. Don’t Jump the Gun on Subsequent Mailings

During your response window, you will notice that the number of responses starts to taper off. While you may be eager to move on to your next mailing, we recommend waiting for any late responses because of the impact it will have on subsequent mailings. As you can see from the 2nd graph above, the 2nd mailing was sent when the response rate reached its lowest. This is to provide the follow-up mailings a revised file of respondent names and addresses before going into production – which can take up to a week or more (depending on the level of printing and assembly required). In so doing you can account for people’s replies from the first mailing and avoid sending them follow-up mailings for a survey they just completed!

 

3. Minimize Post Office Delay

There are specific steps you can take before the returns come in that will help you streamline and expedite the process: 

  • Make sure there is enough money in your business reply account – If you overlook this step, you likely won’t hear about it until after the envelopes have piled up at the post office, and someone gets around to contacting you, costing you valuable time and energy.
  • Make sure your dedicated postal worker is not on vacation – It happens more often than you would think. Postal workers are given various assignments, and there is typically an employee dedicated to handling your company’s reply mail. We recommend you call the post office to ensure that a dedicated staff member will indeed be working on your assignment during your response window.

Survey response mail is among the most exciting yet uncertain parts of survey management. By following these tips, you’ll be better able to estimate your timeline, minimize risk, and account for all of your survey responses.

For more information on survey fulfillment services or any aspect of mail survey management, contact us today!

 

By |2020-03-30T19:31:31+00:00February 27th, 2020|Survey Mailing Services, Survey Research Services|0 Comments

8 Common Survey Bias Errors and How to Avoid Them

When done correctly, surveys are the golden key in social science research – helping us uncover the attitudes and behaviors of a target population like no other platform. As with all scientific research, however, they are susceptible to bias. Bias is a sneaky, subtle characteristic that can creep into any part of your research in the form of leading questions, skewed sample selection, respondent social pressures, and more. In fact, bias is so prevalent in our everyday lives, it is often difficult to spot. 

A primary goal of research is to minimize bias – if not entirely eliminate it. In so doing, results can be trusted as an honest reflection of the attitudes and behaviors of the total target population. Thankfully, survey science has been around a long time and the most common biases are well documented. 

As a leading provider of mail and multi-modal surveys, we help research professionals avoid bias every day. Following are the 8 most common data capture services survey bias errors we encounter, with helpful tips on how to avoid them. 

1. Social Desirability Bias

Respondents have a propensity to answer questions in a way that makes them look good according to social norms. Such topics as taking care of the environment, spending time with one’s children, etc., are ripe for socially acceptable responses that don’t quite match up with reality. This also applies to group norms as well, such as attending church, exercising and more. 

How to avoid it: Don’t use yes or no questions for these topics. Have respondents select from alternatives or use a ranking or rating scale. 

 

2. Acquiescence Bias

Respondents tend to agree and give a “yes” response to most questions, especially if they haven’t given the topic much thought before. For example, would you want your washing machine to have more preset options? Sure, why not. Next question.

How to avoid it: Don’t use yes or no questions for these topics. Have respondents select from alternatives or use a ranking or rating scale. 

3. Question Order Bias

Question order matters. Among the most common culprits of question order bias is “letting the cat out of the bag” too soon. For example, if you are doing a brand awareness survey and mention your brand name too early, you will inadvertently affect how people rate their familiarity with your brand on subsequent questions. This also holds true for response option order. A respondent might remember a choice that appeared in an earlier question and be more likely to select that response on later questions. 

How to avoid it: Use a logical question sequence that goes from general to specific. Manage response order with randomization.

4. Habituation Bias

When a series of questions are worded similarly or use a similar structure, respondents tend to answer in a less engaging way. Instead, many spot the pattern and go on autopilot to get through the survey with minimal energy. This adversely affects data quality, as respondents do not give each question the consideration it deserves. 

How to avoid it: Vary question-wording and keep it conversational. 

5. Sponsorship Bias

When respondents know who commissioned the survey, it can influence responses. Their existing feelings and opinions about the brand or organization can taint even the most general questions in the survey. This is particularly troublesome in product surveys. 

How to avoid it: Do not use any logos on the invitation, survey form or any other collateral. Declare that the survey is being moderated independently of any brand or organization.

 

6. Confirmation Bias

This bias occurs on the researcher side when the survey itself is conducted to confirm a hypothesis, rather than simply gauge opinion. It is particularly common in political circles. Such researchers will give extra weight to responses that confirm their belief, and dismiss evidence to the contrary. In many cases, they will pose leading questions, such as “Don’t you agree that taxes are too high?” rather than a more neutral “Which of the following describes your view on taxes?”  Confirmation bias is a natural human phenomenon, and is not always easy to spot, even within ourselves. It is simply part of the way we process and evaluate information in our daily lives.

How to avoid it: Do not use leading questions. Continually reevaluate responses and challenge them against your preconceptions.

 

7. Culture Bias

Our own cultural experience influences our thoughts and assumptions about other cultures, which can cause unintended bias in research. This phenomenon, known as ethnocentrism, is defined as “judging another culture solely by the values and standards of one’s own culture.” In some cases, the assumptions made can be downright offensive. 

How to avoid it:  Embrace the principle of cultural relativism – that an individual’s beliefs and activities should be understood and evaluated in terms of that individual’s own culture. Have unconditional positive regard and be mindful of their cultural assumptions, too. 

 

8. Halo Effect Bias

People have the tendency to hold an overall impression of something based on only one characteristic. This so-called halo effect can introduce bias on both the moderator and respondent side. For example, a moderator may make an assumption about a respondent-based on a first positive impression. A respondent may respond to a series of questions on a brand based solely on their feeling about one attribute.

How to avoid it: Choose question order carefully and stick to one topic at a time. Continually remind yourself why each question is being asked and hold off analysis until later.

While bias is an integral part of the human experience, it can be minimized in research to deliver an honest reflection of the attitudes and behaviors of your target population. By looking out for these common survey bias errors, you will be well on your way to survey success.

For more information on survey bias or any aspect of multi-modal data collection, contact us today!

By |2020-03-30T19:34:56+00:00January 28th, 2020|Survey Research Services|0 Comments

How to Pick the Right Respondents for Your Survey

In our blog on Choosing the right sample size”, we provided a formula to ensure your target population is represented accurately. Knowing that number early is important for determining your mail quantity and bidding out your project to vendors. However, it is only half the equation in survey sampling. The other half is making sure you pick the right people.  

So how do you choose the participants? 

1. Define Your Target Population

Before you can choose survey participants, you need to define the common binding characteristics or traits of the overall population. For example, “government employees” or “existing customers.” These are often combined with other characteristics: “government employees who use iPhones” or “existing customers who have utilized a particular service.” It is imperative to select the most appropriate target population to satisfy the objectives of the survey. 

 

2. Identify Your List Source

Some survey samples are easier to generate than others. For example, if you are surveying your existing customers, you likely already have everything you need in your company database. But if your target is “Latina women 25-40 who shop online,” you may have some work to do. In this case, you may want to look for available public data or purchase a list from a sample provider. Once you determine the list you need, then you become better positioned to choose a sampling method and pick your respondents. 

 

3. Choose a Sampling Method

There are many scientific ways to select a sample. They can be divided into two groups: probability and non-probability sampling. Probability sampling is any method that utilizes random selection like drawing straws or randomized computer selection. Everyone in a target group has an equal probability of being chosen. It is the preferred method of researchers because it accounts for bias and sampling error. 

But sometimes probability sampling is not feasible, either due to time constraints or list accessibility. In that case, non-probability sampling is used. People must still meet common binding criteria, but they are chosen in such places as a mall or a busy neighborhood. Such samples are often useful but don’t account as easily for bias and sampling error. 

Depending on the needs of your study, you will typically choose from one of the following common methods:

  1. Random SamplingThe purest form of probability sampling. The most basic example of this technique would be the lottery method.
  2. Stratified SamplingIdentifies a subset of the target population such as fathers, teachers, females, etc., and selects them at random.
  3. Systematic SamplingUses every Nth name in a target list, where N is a variable of your choosing.
  4. Convenience SamplingA non-probability method used when only a few members of the target population are available. 
  5. Quota SamplingUses subset criteria like stratified, but doesn’t randomize their selection. 
  6. Purposive Sampling A method that uses predefined criteria with a purpose in mind. For example, gauging the perceptions of Caucasian females between 30-40 years old on a new product but not randomize their selection.

Survey sampling is a critical part of data collection. Your survey provider can help you weigh these options for your survey to ensure you get the quality data you need. For more information on survey sampling or any aspect of mail survey management, contact us today!

By |2020-03-30T19:45:29+00:00January 13th, 2020|Survey Research Services|0 Comments

How to Design a Survey Form (Easy 7 Step Process)

Here’s the truth, answering surveys is not on the list of anyone’s most favorite thing to do- and without a good survey design your survey form might just end up in someone’s trash bin. The good news is, survey design is not rocket science, and it takes only a handful of simple steps and principles for you to make those dull couple of minutes of your respondent’s time a little bearable. So, if you’re ready to create a survey form that will actually be filled out by your desired respondents- check out this non-complicated 7-step process:

  • Step 1- Determine the research goals and list of objectives.
  • Step 2- Think about how you can keep your respondents honest and accurate.
  • Step 3- Keep in mind important Survey Principles before creating your questions.
  • Step 4- Structure questions that will produce all of the information you need.
  • Step 5- Create your Survey Form introduction.
  • Step 6- Select Survey Respondents sample size.
  • Step 7- Choose the best method to collect information.

 

Step 1 – Determine the research goals and list of objectives.

Once the survey questions are finished, review questions to ensure the data will answer your research goals and list of objectives. Eliminate questions that do not contribute to the end-goal and have a colleague review to validate your thoughts and ensure questions are not confusing.  

 

Step 2 – Think about how you can keep your respondents honest and accurate.

How to Design a Survey Form (Easy 7 Step Process)

  • a. If needed, give respondents the option of being anonymous. Some respondents might be hesitant in providing their most honest answer because of embarrassment, fear of judgment or reprimand.
  • b. Do not use too technical terms that might confuse your respondents. Choose the language that mirrors how the respondents truly think and talk regarding the topic.
  • c. Put easier questions first. Allow respondents time to become comfortable in answering the survey, in doing so, they will more likely answer the complex questions later.
  • d. Keep the survey short and simple. Consequently, this will ensure a higher response rate and limit survey fatigue. 


 

Step 3 – Keep in mind key Survey Principles before creating your questions.

How to Design a Survey Form (Easy 7 Step Process)

  • a.  Make sure each question is focused and designed for specific feedback.
    • Don’t use double-barrel questions: “How do you feel about our products and services?”
    • Instead, separate them into two questions: “How do you feel about our product?” and “How do you feel about our services?” These will provide an equal focus on both subjects.
  • b.  Questions should be grouped or ordered according to the subject.
  • c. Questions should be consistent.
    • For example: If you start with 1=low and 5=high, stick with that format.
  • d. Questions should be precise. Use actual numbers. Avoid generic answer choices like “sometimes” and “rarely” in the survey form.
    • For example: “more than 3 times per week”
  • e. Questions should be balanced. The number of positive and negative options should be equal.
  • f. Questions should be complete. Include all possible answers, and make sure there is no overlap between answer options.
  • g. Questions should be bias-free. Construct the questions as objectively as possible.
    • Avoid leading questions: “Can you see why this product was voted best in customer service?”
    • Instead, ask how they would describe their satisfaction level: “Please rate your satisfaction level in using this product.

 

Step 4 – Structure questions that will answer all of the information you need.

How to Design a Survey Form (Easy 7 Step Process)

  • Survey questions fall into two categories:
    • a. Structured or fixed response –  respondents choose from a provided list of answer options.
    • b. Non-structured or open-ended – respondents can fill in their own text or numeric answer.

Common Question Types:

  • a. Multiple Choice – These are questions with two or more answer options and is the most basic type of questions since respondents are limited to choices from the multiple answer options.  These are useful for collecting all kinds of demographic data.
  • b. Rating Scales – Rating scales ask respondents to rate how much they agree with a certain statement using a common scale (e.g. 1 to 5, where 1=low and 5=high). These are useful for gauging respondents’ opinions, attitudes, and behaviors.
  • c. Open-Ended Questions – These are questions with no provided answers options. Respondents answer by writing in their own text. These are great for eliciting responses about attitudes and opinions in a respondent’s own words or having them provide a numeric answer without a suggested range.

If you want to learn more about these question types check out our post on How to Write Great Survey Questions.

 

Step 5 – Create your survey form introduction.

  • a. Advise users of their privacy
  • b. Tell respondents how the data will be used
  • c. Don’t ask personal questions unless necessary
  • d. Give Clear instructions for completing the survey and how long it will take
  • e. If offered, clearly describe the incentive to increase the response rate.

 

Step 6 – Select Survey Respondents Sample Size.

How to Design a Survey Form (Easy 7 Step Process)

  • a. The target population and desired accuracy level will be the basis in choosing the sample size. Target Population is the total number of people you want to understand.
    • For example, you’re doing an employee satisfaction survey, and the company has 1000 employees; then 1000 is the population.
  • b. The Margin of Error – is how much error you can risk. Meaning if you have a margin of error of 5%, and the result of the survey shows 90% of the employees are happy- 85%-95% is the actual number. Simply, 5% is added and deducted on both ends.
  • c. Confidence Level – reflects that the respondents you chose mattered in the results you got. 95% Confidence interval means that you would get the same result 95% of the time and is the most commonly used.

From the chart table, you can determine your target population and then select the margin of error.

 

Population      

Margin of Error

     Confidence Interval
10% 5% 1%       90% 95% 99%
100 50 80 99        74 80 88
500 81 218 476        176 218 286
1,000 88 278 906        215 278 400
10,000 96 370 4,900        264 370 623
100,000 96 383 8,763        270 383 660
1,000,000+ 97 384 9,513        271 384 664

 

Check out this link for The DataForce Sample Size Calculator

 

Step 7 – Choose the best method to collect information.

Below are the most common types of survey distribution. Each survey method has its pros and cons that are affected by the budget, convenience, quality, and other considerations.

  • a. Online Survey– Services like Google Forms, Survey Monkey, Zoomerang, and many others have made online questionnaires very convenient to design and send. You can also use social media or your website to invite people to take surveys. An online survey is the most simple and cheapest survey to manage.
  • b. Face to Face or Telephone Interview – You have to create a script and train people for this kind of survey. This survey requires more effort and budget; however, you get in-depth answers that are most genuine.
  • c. Mail Survey– Despite being old school and having less control, many still opt to use mail surveys because it offers visual quality, looks professional, easy to administer, and not particularly costly.

Take note that you don’t have to choose one. In fact, for best results, it is encouraged to use several modes for survey administration.

Finally, you now have an actionable step-by-step process on how to create your survey form, including guiding principles to structure your questionnaire.  You’re on your way to producing a survey form that will deliver the quality data you need to make that outstanding research. But, perhaps you want to learn more about how to budget for your survey.

Check out this link on Overcoming Survey Budgeting Challenges

For more information on survey design or any aspect of mail survey management, contact us today! We provide outstanding Quantitative Data Collection Services and Paper Scanning Services!

Your Paper-based Data Collection Options Just Got Better

If your mission is to discover answers to objectives, improve performance, or find new ideas and create improvements, then you may need to conduct research and produce statistical analysis to support your mission. In order to do so, you may be faced with the challenge of data capturing from paper surveys and if you don’t choose the right method for your project then you may end up being over budget or facing data quality challenges. To avoid this issue, we will help you review four options to capture data from paper questionnaires, and ultimately, choose the best one for your project. You may be surprised with option 4.

Option 1 – Manual key entry by human operators. 

This option is great for small volumes. Manually entering data into digital format, usually through spreadsheets, can quickly become a time consuming and tedious task. Typically, if you have more than 500 surveys – you will likely be better off automating the data collection, but it really depends on the length of the survey. Manual key entry by human operators is also very prone to errors and many times you will have to use double key entry to ensure accuracy which  will double your labor costs.

Option 2 – Purchasing and maintaining data collection software.

This option can get very costly depending on the number of surveys that you process annually. A decent standalone software that has ICR and OCR capabilities, will be around $15,000 plus you will have to pay for annual support which is typically 18-20% of the cost each year. However, if you have high volume (even just for short periods of time) or need a networked system, it can quickly add up to more than $100,000. This also doesn’t cover the cost of the servers, employees that need to learn and use the software, etc. It adds up very quickly and only makes sense if you are processing hundreds of thousands or a million plus images annually. In short, purchasing and maintaining data collection software can require high initial spend, technical know-how, staff training, and ongoing support cost that is unlikely to be reasonable for a one-time project.

Option 3 – Utilize a service bureau.

This is a great option if you only have a couple of projects a year, or if you just don’t want to manage the process. Just make sure that you understand the bureau’s processes and that they align with your expectations. Does your survey have PII or PHI? If it does, what are their security measures when it comes to your data? Are they using overseas resources? Are their employees trained with HIPAA, are they compliant with HITRUST or any other privacy compliance that you require? Make sure you utilize a service bureau that has the data collection methods, technology, security, and experience to process your research surveys and deliver precise results based on your coding and output specifications.

Option 4 – Rent the software

With the advances in technology, DataForce is able to offer the option for you to remotely and securely scan your surveys into our data collection software that is available to you on a month-to-month, or project-to-project basis. You are able to scan your surveys locally using the “rented” software and any image scanner. You can either use your staff or the bureau’s staff to do the verification. (Verification is reviewing fields that fall out of tolerance so the human operator can apply the established rules.) Using a data collection software on a short-term basis, paper-based data collection becomes pretty painless and you only pay for the images that are processed. When a software rental option is utilized, you can select the most appropriate data capture methodologies and processes to complete your survey project on time and in budget.

DataForce will provide you with the training, survey scanning software, and support to gather the information your organization needs in the convenience of your own premises. No long term commitments or costly annual maintenance Learn how!

The Definitive Survey Design Checklist

 

In our previous blog, we outlined the basic principles of questionnaire design for writing great, effective survey questions. Of course, your overall survey design will include more than just the questions. You will also need to include a survey title, provide instructions and add a thank-you closing statement, as well as review and test the effectiveness of each element before launch. If only there were a simple survey design checklist to follow to ensure you didn’t miss anything.

Well, you’re in luck. We’ve created a handy checklist of all the essential elements to include in your survey, including style and content considerations. We present to you … the Definitive Survey Design Checklist:

Objectives

____ Your survey objectives have been documented

____ You are clear on how you plan to use the data

Survey Design

Introduction

____ Your survey has a clear title

____ You explain the purpose and importance of your survey

____ An estimate of survey duration and length of questionnaire is included

____ You included a confidentiality guarantee or ask permission to share their responses and identify with whom

____ You provided brief company background information

____ You referred to your survey incentive (if any) and provided information about it

Questions

____ The survey begins with a simple question

____ The questions go from general to specific

____ Sensitive questions appear toward the end of the survey

____ Sensitive questions include a “prefer not to answer option”

____ Demographic data appears at the end of the survey

____ Questions are organized by topic

____ All questions relate to your objectives

____ All questions are simple and concise

____ Jargon, acronyms and technical terms have been avoided

____ Response options include all possibilities, using “other” or “none” as necessary

____ The majority of questions are closed-ended for easier data analysis

____ Directions on how to answer are placed before each question

____ Rating scale questions include the rating scale before the question

____ Rating scale questions include a midpoint answer and an equal balance of positive and negative choices

____ Response options are placed vertically when possible, except for tabulated questions

____ Multiple choice questions display the most positive answer first

____ Open-ended questions are voluntary

Closing

____ You thanked your respondents

____ You offered them the possibility to receive the results (if possible)

____ You provided details on receiving the incentive

Final Review

____ You have sent the survey to colleagues or friends to validate wording and timing

____ You have pre-tested the survey by sending to a small group of respondents in the target population

____ The survey takes less than 10 minutes to complete

____ You have made appropriate revisions to address any issues uncovered

By following this guide, you will be well on your way to survey research success. For more information on survey design or any aspect of survey mail management, contact us today!

 

 

 

By |2019-03-20T10:48:48+00:00December 20th, 2018|Survey Research Services|0 Comments
[activecampaign form=7]