Survey Research Services

DataForce Survey Research Services Information, Tips, and How-tos

How to Write Great Survey Questions

When starting a survey project, most people look forward to the fun, creative part of writing the questions. However, it doesn’t take long to realize that writing great survey questions is not as easy as it looks. Questionnaire design is more science than art – requiring critical attention be paid to question and answer order, structure and phrasing to ensure you get the reliable, quality feedback you are looking for.

A simple question, such as “How much did you enjoy the program?” could wreak havoc in your results, because it is inherently biased towards a positive response. “How did you feel about the program?” would be a more effective approach. Other pitfalls include asking multi-part questions, having overlapping answer choices, or asking the more difficult questions too early.

But have no fear. Outlined below are the basic principles of questionnaire design, along with some helpful tips, that will have you writing great, effective survey questions in no time:

Before You can Start Writing Great Survey Questions 

    • Know your objectives. Write down the purpose of your survey scanning, what information you need, and how you plan to use the data.
  • Work backwards. Make a list of the specific answers you need first, and then use that to drive your questionnaire.

Basic Guidelines for Writing Great Survey Questions

    • Keep questions focused. Make sure each question is designed for specific feedback. Avoid double-barrel questions like “How do you feel about our products and services?” as some respondents will focus on products and others on services. Instead, separate them into two questions.
    • Put easier questions first. This will increase participation and establish trust. By getting comfortable with the survey research by answering a few less complex questions first, your participants will be more likely to answer the more complex or sensitive questions later.
    • Organize by topic. Similar questions should be grouped together so the questionnaire flows naturally.
    • Keep it short and simple. Questions should be short, focused, and easy to answer. This will ensure a higher response rate and limit survey fatigue.
    • Be consistent. Use uniform rating scales, word choices and definitions throughout your survey. If you start with 1=low and 5=high, stick with that format.
    • Be precise. Avoid generic answer choices like “sometimes” and “rarely”. Use actual numbers instead (e.g, “more than 3 times per week”).
    • Be balanced. Provide an equal number of positive and negative response options.
    • Be complete. Include all possible answers, and make sure there is no overlap between answer options.
  • Eliminate bias. Try to construct the questions as objectively as possible. Avoid leading questions like, “Can you see why this product was voted best in customer satisfaction?” Instead, ask how they would describe their satisfaction level.

Common Question Types

Survey questions fall into two categories: Structured (fixed response) where they choose from a provided list of answer options and Non-structured (open-ended) where they can fill in their own text or numeric answer. Both are extremely useful, depending on the type of feedback you need.

Following are the most commonly used question types:

Multiple Choice

These are questions with two or more answer options. These are useful for collecting structured responses.

Single Response Style (select one answer)

Example 1: Do you smoke?     Y / N

Example 2:  If yes, how many cigarettes do you smoke per day?

use multiple choice to write great survey questions   

A common pitfall here is missing a possible response. Depending on your question, you may need to add a choice called “none”, or if you would like additional details, you could try an “other” option with space for a written response. You also want to make sure there is no overlap, such as using 10-20, 20-30, etc. in the previous example, which would clearly taint the results.  

Multiple Response Style (you may select more than one answer)

Example 1:  What is your race? (check all that apply)

use multiple response style to write great survey questions

Rating Scales

Rating scales ask respondents to rate how much they agree with a certain statement using a common scale (e.g, 1 to 5, where 1=low and 5=high). These are useful for gauging their opinions, attitudes and behaviors. When using rating scales, it is important to make sure you have a neutral option and a balanced, equal number of positive and negative responses. Scales most commonly use 5 or 7 options.

Example 1: The teacher was knowledgeable.

Example 2:  How would you describe your experience navigating the instruction manual.

use rating scale to write great survey questions (sample 2)

Common pitfalls here include being inconsistent with your scales (leading some respondents to answer incorrectly) and asking leading questions, such as, “We pride ourselves on our easy-to-use manuals. How easy was our manual to read?”

Ranking Scales

These ask respondents to rank a list of items in order (e.g, from favorite to least favorite, or most important to least important). It is recommended that you use these with caution. They are known to be reliable at determining first and last place, but not so much the fuzzy middle, as respondents often have to choose a pecking order for items that are essentially of equal value to them.

Example 1: Please rank the following customer service features in order of most to least important when contacting our agency by phone (1=most important, 5=least important)

use ranking scale to write great survey questions

Open-Ended Questions

These are questions with no provided answers options. Respondents answer by writing in their own text. These are great for eliciting responses about attitudes and opinions in a respondent’s own words, or having them provide a numeric answer without a suggested range. The downside is it requires extra time, can cause survey abandonment, and makes data collection and analysis more challenging.

Example: Name two ways we could have improved your customer experience today?

Questionnaire integrity is critical for getting quality data. By following these tips and guidelines, you will be well on your way to success. 

For more information on question and survey design or any aspect of survey mail management, contact us today!

By |2019-03-20T10:51:08+00:00December 6th, 2018|Survey Research Services|0 Comments
Read More

When to Do a Multimodal Survey?

Multimodal or mixed-mode surveys are research surveys that use two or more forms of communication to reach respondents (e.g, telephone and email). In today’s increasingly complex, interconnected world, we now have ways of communicating that didn’t exist even a few years ago. The list of channels seems almost endless nowadays, including regular and express mail, email, online, social media, mobile (text, instant message), scannable paper, telephone, kiosk, tablet, in-person, video, and more.   

So how do you know which channel or combination of channels is right for your survey project? The answer lies in the target audience you are trying to reach (as well as time and cost considerations). For example, older respondents are typically less trusting of online channels and can be reached more reliably by landline telephone and regular mail. Millennials may not even have a landline and would be more receptive to an email or a text message. Teens are generally receptive to mobile and social media. One constant to be mindful of, however, is that everyone has a physical address where they live – making mail a preferred channel in almost any multimodal research effort.

Not only do various target populations have preferred forms of communication, but there are also subsets within them that prefer something else entirely. Mobile phone call vs. text message among Generation X’ers is a prime example, in which someone’s preference is highly personal.

You should do a multimodal survey if you have a target population with varied respondents or hard-to-reach respondents.

Not only can a mixed methodology approach to data collection help you reach more respondents, but it can also help you maximize response rate. That’s because multiple channels give you more opportunities for follow ups, reminders, and options to complete the survey in a format that suits them.

As a leading provider of mail and multimodal surveys, we manage mixed methodology research every day, including planning, production, distribution, fulfillment and data collection. That said, our most popular service by far is Mail to Online. In this strategy, respondents are notified by regular mail and given instructions to complete the survey online. Respondents then have the option to print out or request a paper survey and complete a hard copy or complete the online survey.

Multimodal survey planning and execution requires a high level of expertise to run seamlessly. Your survey provider can help you manage all the details, including which channels to use, along with the projected cost and timeline.

For more information on multimodal surveys or any aspect of survey management, contact us today!

By |2019-03-20T10:54:13+00:00November 21st, 2018|Survey Mailing Services, Survey Research Services|0 Comments
Read More

The Most Important Skillsets for In-House Survey Projects

If you are planning to effectively and efficiently handle a large-scale survey project in-house, knowing the questions you want to ask your target respondents is only the beginning. You also need to have access to a wide range of specialized skillsets. This is because practical considerations are certain to arise throughout the project that will require smart planning based on experience.

Here is a quick overview of a few of the most important skillsets involved in completing a large-scale survey project in-house.

Survey Design Skills

The core objective of your survey project is to obtain answers to questions that will help you analyze the thoughts, beliefs, actions, or experiences of your respondent population. To reach that objective, you must compose questions that will elicit useful responses.

During the survey design stage, it is important to consider more than the research objectives of your survey. There are other practical considerations relating to the efficiency of the project. Page count is an excellent example. If your survey is very complex or has so many questions that it must be printed in a large, multi-page booklet, you are jeopardizing the success of your project.

Large booklets create enormous challenges. They are difficult to design, print, and distribute. Additionally, they require extra effort to dismantle and collate so that they can be processed through your data extraction system. The greater the number of pages, the greater the complexity and room for error.

Graphic Design Skills

Another important skillset is graphic design. Your in-house designer must be proficient in Adobe InDesign, which is the most popular software for laying out surveys. Even though they may be a great graphic artist, their general skillset will only get them so far. They must also have experience in dealing with the many design choices that will impact your respondent’s experience while taking the survey as well as back-end quantitative data collection issues.

The layout of the survey must make it easy and intuitive for those completing the survey to fill in their responses. Even simple things like requesting a date of birth or phone number are a critical design issues. The choice to use a line to collect this information is problematic.

That’s because it is difficult for scanning software to read information formatted in a line with acceptable accuracy. The better design decision is to create a series of boxes that require respondents to place one letter or number in each box. This design option encourages respondents to write neatly. It also results in a more legible survey that is easier to review manually.

Printing Skills

The design of the survey affects the complexity of printing the survey. Remember that one of your goals is to automate data collection. Therefore, issues such as the number of pages, the layout of each page, and the placement of design elements are extremely important. For projects that require optical mark recognition (OMR), barcodes, or pre-slugging for accurate survey scanning and tracking, small printing errors have big consequences.

Logistics Skills

Managing the distribution, collection, and storage of your surveys presents an enormous logistical challenge. Once you’ve printed your surveys and support materials, you must then assemble the survey packets and distribute them to your respondents. You must also be prepared to receive the surveys when they are returned. The logistics skills involved include collating, packaging, labeling, addressing, bulk mailing, return-mail handling, warehousing, inventory tracking, and follow-up mailings.

Data Extraction Skills

Assuming that you have the necessary OMR and image-scanning equipment in place, you’ll also need to follow strict quality control measures to ensure the accuracy of your data. The data extraction process requires reliable data output formats, tables and rules, exceptions, electronic verification, manual verification, and image indexing to facilitate easy search and retrieval.

Do You Have Access to All of the Skillsets You Need?

As you prepare for your large-scale survey project, consider making a thorough assessment of your team’s skillsets. If you don’t have easy access to all of the skills you need to complete your project effectively and efficiently, it’s time to search for a partner that can help. Doing so will help your organization maximize its return on investment in time and money.

For more information on paper scanning services, forms processing services or any aspect of survey mail management, contact us today!

 

By |2019-03-20T11:17:40+00:00March 7th, 2018|Data Capture Services, Incentive Fulfillment Services, Survey Mailing Services, Survey Research Services|0 Comments
Read More

Overcoming Survey Budgeting Challenges

Establishing a budget for your survey project is challenging. There are many facets to consider. It’s tempting to pull a number out of the air that sounds reasonable and then fit the costs for your survey within that pre-determined budget.

That’s an easy way to set your budget. But, it’s far from the best way.

It’s more effective for you to meet your budgeting challenges head-on. Take the time you need to consider your survey objectives and the costs associated with each step of the project. Only then will you be confident that your budget supports the success of your project.

Work Backwards from Your Expected Response Rate
Your expected response rate is an important budget driver. It impacts your sample size as well as the number of surveys you’ll need to distribute to your targeted respondents. As you know, your response rate will vary based on several factors. A few of these include:

• Questionnaire length
• The complexity of the questions
• The importance respondents place on the survey
• Availability of incentives
• The level of personalization

Every survey project is different. However, every project benefits from achieving the highest response rate possible. Here is an overview of a few factors that influence both your budget and your response rate.

Buying Lists

If you don’t already possess a list of targeted respondents, you’ll need to purchase one. The size of your list will depend on how large of a sample size you need as well as your expected response rate.

Printing Survey Materials

The costs associated with printing your survey materials represent a significant portion of your budget. Consequently, printing is a good area to focus on in your attempt to control costs. Depending on the needs of your project, you can make printing decisions that could result in spending as little as five cents or as much as five dollars per survey. How many copies you need to print is one factor. The aesthetics of those surveys is another factor. You can print one, two, three, or four-color surveys. You must also decide on the number of pages, binding, and the weight of the paper. Every decision affects your costs.

Multiple Mailings

Depending on the methodology of your project, it may be beneficial to conduct multiple mailings. Mailing survey packets, follow-up post cards, and replacement survey packets add to your overall cost. You must be careful to balance your expenses with the impact you believe each mailing will have on boosting your response rate.

Consider Offering Incentives
To reduce printing costs and expenses associated with multiple mailings, you should consider offering an incentive to increase your response rate. Research in this area has shown that incentives can double response rates. If you choose to offer incentives, you’ve created another budgetary challenge. First, the price of the incentive increases your budget. Second, research shows that you can expect diminishing returns. So, you must be careful to choose an incentive value that balances cost and effectiveness.

Build Your Budget Around Your Survey
The best approach to overcoming your survey budgeting challenges is to base your budget on the cumulative projected costs of your survey. That’s because you’ll be starting with a clear view of your survey objectives and you can decide how to reach them based on the actual costs of your many options. If you instead start with a specific budget amount, you may find that you’ve picked a dollar figure that bears no practical relationship to your survey objectives. By constraining yourself with a budget at the outset, you shift your focus away from what matters most.

To help you build your budget around your survey, it is crucial that you consult with your survey partner early and often. Your partner will not only provide you with cost information, they will also offer recommendations that will help you save money while achieving your goals.

By |2019-03-20T11:11:29+00:00March 7th, 2018|Survey Research Services|0 Comments
Read More

Good, Fast, Cheap – Pick Two

The other day I got into a heated discussion with a colleague about this quote. You can Google it and it will return 100’s of pages and images. It appears that there are even businesses that post signs, telling their customers they can only have two.

In my humble opinion, any organization that communicates and/or promotes this quote is preparing you, the consumer, to either:

a. Be prepared to pay a lot (Fast/Good)

b. To be able to say “I told you so” when it didn’t go so well (Fast/Cheap)

c. Have the perfect excuse as to why your product/project is 2 weeks over due (Cheap/Good)

Personally, I would never do business with someone that would put themselves into a box. I think we should always be striving for the ‘impossible’ utopia.

Don’t get me wrong, if you are looking for a Cadillac with a Pinto budget; you will never be happy. Make sure that are you working with a vendor that you trust, understands your objectives – if you don’t trust them to give you sound advice; it is time for a change. The right relationship shouldn’t feel like a customer/vendor relationship; it should be a partnership.

Don’t limit yourself, we should always be thinking outside of the box on how to do things better, faster and more efficiently.

By |2019-03-20T11:12:56+00:00March 7th, 2018|Data Capture Services, Incentive Fulfillment Services, Survey Mailing Services, Survey Research Services|0 Comments
Read More

Choosing the Right Sample Size

Choosing the right sample size is vital to accurate survey outcomes. With the right number of survey respondents, your target population will be represented with acceptable accuracy.  Although it may seem like a simple undertaking, you must put a lot of thought into determining your survey’s most effective sample size.

In most cases, the larger your sample size, the more confident you can be that the responses you receive reflect the total target population. However, there is another side to that coin. Deciding on a larger sample size requires you to distribute more surveys, and that drives up the cost of your survey project.  

It’s not as simple as randomly selecting a large sample size and assuming that will equate to successful response rates. The pitfall of using no forethought when selecting a sample size is that you may select a sample that is not statistically significant or will cause you to exceed your budget. To arrive at your optimum sample size, you must start by asking a few key questions.

1. How Large Is Your Target Population?

Choosing the Right Sample Size - How large is Your Target Population

You can begin to understand your sample size needs by first determining the size of the total population of possible respondents. If you want to understand something more large-scale, such as how grocery shoppers are using mobile devices, you will need to use a larger sample size. That’s because there are millions of grocery shoppers who use mobile devices. On the other hand, if you want to assess a smaller, more specific group, such as the opinions of the faculty at a particular university, the population is limited to a few thousand or even a few hundred.

2. How Accurate Do You Need to Be in Choosing the Right Sample Size?

Because you are only using a small sample group out of a larger target population, you must now make some decisions about accuracy. Your accuracy will depend on choosing an acceptable margin of error and accepted confidence level.  

Your margin of error measures how well the responses from your sample reflect the opinions of your entire target population. A commonly chosen margin of error is 5%. You must also keep in mind that lower margins of error require a larger sample size.

Your confidence level represents the degree to which your sample is an accurate sample of the population. If you decide your project requires a 95% confidence level, you are declaring that if you repeated your survey, you would get the same results 95% of the time. Higher confidence levels require a larger sample size.

3. What Is Your Expected Response Rate?

Arriving at an answer to the first two questions is relatively straightforward. First, determine the size of your target population. Then decide how accurate you want your survey results to be.

Now, on to the challenging part: estimating what your response rate will be. Estimating your response rate is often difficult; there are many factors that contribute to a respondent’s decision to complete and return your survey. The list of factors includes:

    • Questionnaire length
    • Question complexity
    • Importance of the survey to the respondent
    • Level of respondent interest in the survey
    • Availability of incentives

Fortunately, you are in control of most of these factors. Depending on your target population and the decisions you make about how you will administer your survey, your response rate can range from 1% to 60%. Lower expected response rates require a larger sample size.

For more tips on Survey Design, check our blog post on How to Write Great Survey Questions. 

Working with an Experienced Survey Administration Partner in Choosing the Right Sample Size

Your sample size has a direct impact on the validity and cost of your survey. Consequently, it’s crucial that you get that number right. Once you have determined the right sample size for your survey, there’s still more work to do. How will you distribute the surveys? How will you gather them and analyze the outcomes?  

This is where your survey administration partner can add value to the process. When you work with an experienced vendor, they can consult with you about the nature of your survey research and suggest choices that will help you achieve your goals in an efficient and cost-effective manner.

Your partner can help you measure your target population and decide on an appropriate margin of error and confidence level. In addition—although no two survey projects are ever exactly alike—your partner can rely on their experience with working with a wide range of organizations and projects to help predict what your response rate might be.

When you are planning your next survey project—and trying to understand your optimal sample size—consider getting help from an experienced survey administration partner.

If you’d like more information on incentives, project consulting services, survey research services, incentive fulfillment or survey mailing services in general, contact us today!

By |2019-05-24T19:47:16+00:00March 6th, 2018|Survey Research Services|0 Comments
Read More

4 Ways to Reduce Vendor Management Challenges

Successful survey projects rely on a wide range of specialized skills and technologies. Because very few organizations have the capabilities to handle the entire survey administration in-house, it is typically necessary to outsource the work to multiple vendors.

Depending on the complexity of your project, the number of vendors you may need to manage can range from a handful to dozens.

Your basic list of tasks to outsource can include:

  • Graphic design
  • Printing
  • Document tracking
  • Fulfillment and mailing
  • Warehousing
  • Data extraction

That’s just the beginning.

The more complex the project, the more vendors you’ll need to coordinate. For instance, if you are offering cash incentives, you’ll need to work with a bank. If your project requires special items such as containers for samples, selecting and managing your growing group of vendors becomes an even greater challenge.

How to Choose a Reliable Vendor

Regardless of the number of vendors you’ll be managing, you need a process for selecting a vendor that you can rely on to help you complete your project as efficiently as possible. Here are three important considerations when selecting vendors for your survey project.

Ask for References, and Call Them

Be sure to ask each vendor for at least two current references as well as a reference from a recent customer that is no longer working with them. Then call them.

When you call the current references, ask them about the vendor’s capabilities, customer service, responsiveness, and reputation for meeting deadlines. If there are any red flags, identify them before contracting with the vendor.

It’s likely that the vendor will put you in contact with current references that are satisfied with their service. But calling a vendor’s former customer gives you an even better perspective, particularly if that customer is less than two years old. When you speak with a former customer, you can dig deeper. They may still give a glowing reference, but it will be more objective.

Monitor Each Vendor’s Financials

You don’t want your survey project to fall apart because one of your vendors suddenly goes out of business. To help avoid a surprise like this, be sure to request a copy of each vendor’s audited financials and review them closely. If you work with the same vendor for many years, take the time to compare their financials year over year. Negative trends will alert you to future problems and give you time to discuss your concerns with the vendor or begin the process of searching for a more financially stable vendor.

After You Select a Vendor, Stick with Them

Selecting the right collection of vendors for your survey project requires an enormous amount of time and effort. Once you’ve found a vendor that meets all of your needs, it is important that you stick with them.

Ideally, you want to build a long-term relationship with your vendors so that they understand how your organization works and they can become a valuable and loyal resource. Whenever you change vendors, you lose that loyalty and have to start the relationship-building process all over again.

Choose One Partner Than Can Handle Your Entire Survey Project

As the number of required vendors rises, the time and effort necessary to orchestrate the activities of multiple vendors will increase outsourcing costs, and strain your resources. One of the key decisions you must make when planning your next survey project is whether to work with multiple vendors, or to select a partner that can handle all aspects of the survey administration process.

By selecting a partner that can act as your single point of contact throughout the project, you’ll save time and money. You’ll avoid having to go through an exhaustive selection process to narrow down the field of vendors for each component of your survey. You’ll also escape the headaches involved with coordinating interactions between each vendor.

Even though your partner may not be able to internally handle all of the tasks your survey research requires, you can rely on them to manage any necessary sub-vendors. Your partner will already have relationships with vendors they trust, and they will have worked out effective processes for coordinating the complex logistics involved with managing multiple vendors.

An effective survey administration partner is flexible enough to offer you the exact services you need. After consulting with you, they must be able to tailor their services and the administration process to the precise needs of your project.

When faced with the frustrations of managing multiple vendors, choosing a single, experienced survey administration partner is the solution to your vendor management

If you’d like more information on project consulting services or data capture services in general, contact us today!

 

By |2019-03-20T11:21:10+00:00January 11th, 2017|Survey Research Services|0 Comments
Read More

DataForce is an industry-leading data collection company specializing in mail survey and study management for all business sectors with a focus on mission-driven companies in education, government and healthcare.

Copyright 2018 DataForce | All Rights Reserved
connect with us
linkedintwitter
Toggle Sliding Bar AreaGET QUOTE