Surveying the friendly skies

Most research projects have their procedural intricacies but try these on for size: Each month, imagine distributing 192,000 surveys - in nine languages - to people traveling to 40 different countries. The surveys have to be shipped from a central location to destinations all over the world and returned on a timely basis. And, for toppers, the people you're surveying happen to be 30,000 feet in the air.

If that kind of a project is on your plate, it's helpful to be in the airline business.

For its in-flight customer satisfaction tracking program, Chicago-based United Airlines surveys passengers on some 900 flights per month using a four-page scannable form. The survey covers passenger satisfaction with the entire air travel process, from reservations to airport service and several aspects of the flight, including the flight attendants, the meal service and the aircraft itself.

Early in the flight, passengers are asked to participate in the survey. In exchange for their cooperation they are entered into a sweepstakes for two free round-trip tickets. Flight attendants distribute the survey forms and pencils. Depending on the length of the flight and time of day, the surveys are either handed out at the beginning of the flight or after meal service and collected later by the flight attendants.

"The biggest challenge is coordinating the logistics of the project, because you have so many people involved along the way, both internally and externally," says United's Alex Maggi, senior staff analyst, market research. "It begins here at the research department but it involves people in our freight area in the Minneapolis airport and customer service reps at both the departure and arrival airports. We also rely heavily on the participation of the flight attendants."

In addition, United has a close working relationship with National Computer Systems (NCS), the Edina, Minn., firm that handles the printing and processing of the survey forms and is an integral part of the survey distribution system. "There are many parts of the project that must be completed on a timely basis so we have to have a real partnership and be responsive to each other to make it all fall into place," says Sharon Harding, NCS project administrator.

The cooperation between United and NCS extends all the way down to the individual form level. Because of the huge amounts of material involved in the program - the thousands of survey forms, the hundreds of survey containers - United and NCS have worked together to make the process as efficient as possible. The cases and cartons that the surveys are shipped in are reusable and NCS workers sort and save unused survey forms for use on other flights.

Busy life

The life of a United in-flight survey form is a busy one. After the questionnaires are designed and proofread (the eight foreign language versions are checked for accuracy by translators) they are sent to NCS, where the surveys are printed, collated and shrink-wrapped. NCS then stores the surveys until they're ready for shipping.

Each month, United selects flights to survey and transmits this information electronically to NCS. "We run a program which randomly selects a percentage of all of our flights system-wide, 1 percent of domestic flights and 7 percent of international flights," Maggi says. "We transmit this file electronically to NCS. It provides NCS with different types of information about the flights that are going to be surveyed. From this file they can see the date of the flight, the flight number, the aircraft type, how many surveys are needed and the distribution of the surveys in the different languages, if necessary."

Using these specifications, NCS packages the survey forms, instructions and pencils into flight cases and delivers them to United Airlines Cargo at the Minneapolis airport for distribution to the proper airports. "Depending on the region to which the surveys are being sent, they have specific ship dates. So for each week of the survey there is a date by which they have to be packed and shipped. NCS generates a packing list and on the appropriate dates they pack the proper amount of surveys into flight cases and cartons corresponding to the flights we've selected and then affix labels indicating the segment to be surveyed," Maggi says.

When the survey packages arrive at their respective destinations, they're picked up by a United Airlines manager or customer service representative who makes sure they get delivered to the flight crews. "Each month the airport stations receive a list of the flights that will be surveyed that month so they know when to expect them. The flight attendants are also notified in advance which dates they'll get surveys," Maggi says.

Once the surveyed flight reaches its destination, the completed forms are repacked and handed off to the receiving United customer service representative, who ships them back to United's Minneapolis cargo location.

Three times a week, NCS picks up the completed surveys. Twice a week they drop off the next shipment of surveys to be flown out. The survey forms are scanned and NCS transmits the data electronically to United on or about the tenth of the following month, along with status reports on the flights of that month.

Working with the data

After the data arrives at United, it is converted into Quanvert, a statistical program from Quantime Corp. Maggi also uses SPSS to analyze the data.

The United research department fields all kinds of requests for data, he says. "People are eager to get their hands on any piece of information they can. They like to break the data down to the micro level wherever possible and we're able to do that.

"It's a challenge to take such large volumes of quantitative data and make it meaningful to our audience. We're constantly soliciting feedback from those who use the information to make it truly meaningful. We always have the users of our data involved in the survey design process as well as the report design process."

The marketing research department at United issues a monthly report summarizing the customer satisfaction data for about a hundred people worldwide, including airport, country and regional managers and executive management and others at United's headquarters. Segments of the data are also available on-line.

"Executive management uses data from the in-flight surveys to look at competitive performance. Here within marketing we analyze the data and use the results for strategic planning, product positioning and target marketing. Finance uses customer satisfaction data to measure the success of its product investments. If they have spent money to improve service they want to see that there has been a tangible improvement," Maggi says.

The research data is also used to help track performance and measure success at achieving goals. "Our airport managers are interested in all ratings related to ground service, including speed and efficiency of check-in, the friendliness of customer service reps and line waits. The reservation area is interested in findings on service areas such as hold times on the phone and helpfulness of the agents. The in-flight supervisors who manage the flight attendants want to know how the flight attendants are being rated on the various aspects of their service, not just their overall rating but everything from timely delivery of the meal and beverage to friendliness and professionalism."

Because the data can be linked to operational data such as arrival and departure times and number of passengers, the United researchers can dig deep to answer questions from in-house customers, Maggi says. "We have often used the data to identify the reasons why some ratings might differ from one airport to another or one segment to another, by looking at customer mix, by linking survey data to operational data. For example, we can take ratings for a given flight and link them to the on-time performance of the flight in that market and we can show that when on-time performance went down so did the ratings in specific categories."

Customer oriented

Maggi says that since the airline became employee owned in June 1994, it's become much more customer oriented. As a result, the research department is busier than ever. "It's been truly amazing the amount of attention [the market research department] has been getting lately. We constantly receive inquiries about research results from people at all levels of the organization. We've also greatly increased the scope of the in-flight customer satisfaction tracking program."

By tripling the size of the sample in the past year and a half, the database more accurately reflects customer demographics, Maggi says. "We did that to better ensure a representative sample of our customer population. In the past there were times when we were not able to get as many completed surveys as planned and that affects your sample sizes, so you would have to put an asterisk next to some of the data. Whereas now we have a much more complete database and the data is much more reliable."