Lessons from the College Board

Editor’s note: Frank X. Allen is a Chicago-based information technology and business writer.

Market researchers are increasingly turning to the Internet to collect information, often in the form of online surveys. As the College Board found out, to achieve the best results, surveys should keep in mind the needs of respondents.

The College Board, a national, nonprofit association best known for creating and sponsoring the SAT college entrance exams, completed a highly successful online survey of 3,200 admissions officers at colleges and universities across the country. The results of this survey were published in a white paper titled “Trends in College Admissions: A Report of a National Survey of Undergraduate Admissions Policies, Practices and Procedures.”

Working with Thomas Technology Solutions, Inc. (ThomasTech), a Horsham, Pa.-based systems integrator that provides content management and electronic publishing solutions, the College Board delivered a survey that:

  • combined paper- and Web-based versions produced from a single source;
  • accommodated virtually all browsers;
  • let respondents save a partially completed survey for later completion;
  • included a “send” button to alert the College Board about completed surveys;
  • ensured security of data; and
  • provided around-the-clock access to phone support.

Combine paper- and Web-based versions

It may sound contradictory, but to get the best results from a Web-based survey, combine it with a paper-based version. That’s the advice of Renee Gernand, senior director of guidance services for the College Board, and John Mullen, vice president of information services at ThomasTech.

Of the 3,200 admissions officers who received the survey, more than 1,400 (43 percent) completed it. The combination of paper and online was key to that high rate of response. “People like to see the survey first before they go online,” says Gernand. “They want to know what they are getting into.”

Having the survey on paper allows respondents to review the questions, collect information, and estimate how long it will take them to complete the survey. “Providing the survey on paper and online also gives respondents a choice, which makes them more likely to cooperate,” says Mullen. In the end, fully two-thirds chose to complete the survey online.

The College Board survey began with mailing paper copies of the survey. In a cover letter, the admissions officers were invited to take part in the survey and were given the choice of completing the survey on paper or online. “We designed the Web site so that it would mimic the paper version,” says Mullen. Because the Web-based survey looked like the paper version, respondents were less likely to be confused and more likely to complete the survey online.

But getting an online version of a survey to look like the paper version can be a real challenge. There are many different Web browsers in use and each browser has different capabilities.

When developing a Web survey site, be sure to ask your developer about the browsers your customers may use. Gernand was surprised by the number of different Web browsers - some of them quite old - used by colleges and universities. That made it crucial to design the College Board survey Web site in such a way that it would accommodate all potential users’ browsers. “Unfortunately, handling this variety of browsers cannot be done with an off-the-shelf Web development application,” says Eric Jacobs, systems analyst at ThomasTech. “We had to do a significant amount of server-side coding to accommodate the different browsers.”

Let respondents save partially-completed survey

One of the biggest problems with online surveys is that respondents abandon the survey partway through. At 16 pages in length, the College Board survey was long enough to require special consideration. The reality of the College Board survey was that numerous people needed to provide input. In response to this need, ThomasTech designed the Web site so that the respondents could fill out part of the survey, save it, and then come back later to add more information. Key to this design was dividing the survey into sections of a manageable length with a “save” button available at any point in the survey.

Mullen and his team designed the survey so that completed portions appeared in a different color on the index page, enabling a returning respondent to quickly move to uncompleted or partially completed sections.

Include a “send” button

The period during which individual college admissions officers worked on their surveys ranged from a single day to several months, depending upon how long it took for them to gather their information. To keep the College Board apprised of the progress of the survey, ThomasTech provided weekly reports to Gernand about who had logged into the site, who had begun the survey, and who had completed the survey.

Those who had started the survey but hadn’t completed it were sent periodic reminders prompting them to finish, resulting in a higher rate of completion.

In the end, Gernand wanted to know when respondents were finished so their information could be tallied into the survey totals. By including a “send” button feature, Gernand and her staff were able to distinguish between those who were done and those who still needed more time to complete the survey. “Be sure to give them a mechanism that lets you know when they have finished the survey, otherwise you don’t know they are done,” Gernand advises.

Ensure data security

Survey participants want to know that the information they are providing is safe from unauthorized viewers. For the College Board, these security concerns were heightened because partially completed surveys needed to remain accessible for long periods, as the survey participants worked to complete them.

In the College Board survey, that security was outlined up front. “Along with the paper survey, we sent a cover letter to each administrator with a user ID and password,” Mullen says. “It was important to let them know right away that the process would be secure.”

Provide trained help desk support

No matter how well a Web site is designed, users will have questions. That’s why it is important to provide them with a toll-free number - and trained representatives at the other end - that they can call to obtain answers. With the College Board survey, respondents logged in from Maine to Hawaii. As a result, College Board used ThomasTech to provide help desk support around the clock. “You need very good help desk support for a survey like this,” Gernand says. “Without it, you can lose a lot of respondents.”

Keys to survey success

The Web offers enormous possibilities for handling large-scale surveys. As evidenced by the College Board survey, even long and complex surveys distributed to a large number of people can be successful, as long as some basic tips are followed.

  • Keep the needs of your respondents in mind by providing a paper-based version of the survey, accommodating a large variety of browsers, allowing respondents to save partially completed surveys and ensuring the security of the respondents’ information.
  • Have the technical proficiency to deliver your survey. Off-the-shelf solutions may not accommodate many browsers and may restrict your ability to design Web pages to mimic your paper-based survey. And be sure to provide your respondents with the best possible phone support.

The experience of the College Board demonstrates that to produce maximum response rates, it is important to keep the needs of your survey respondents in mind. “I can’t overstate the importance of working with an experienced Web-development consultant,” says Gernand. “It is critical for the consultant to understand how the survey user thinks.”