A global perspective

The Global Airline Performance (GAP) study is a joint venture between P. Robert and Partners (PRP), a Swiss research firm, and the London-based Aviation Information and Research unit of IATA, the International Air Transport Association.

The syndicated study uses a two-part survey to measure the opinions of air travelers on 22 airlines departing from 30 airports in North America, Europe, and Asia. Each year, 240,000 passengers are interviewed. Depending on the airline routes being researched, the survey can be conducted in seven languages: English, French, Dutch, German, Swedish, Chinese, or Japanese.

GAP Transatlantic (operating since July 1999) covers 16 airlines, 15 airports in North America and 10 in Europe. GAP Transpacific (October 1999) covers 12 airlines, 15 North American and four Asian airports. GAP Europe to Asia (April 2000) encompasses 12 airlines, six European airports and four Asian airports.

No airline or airport personnel are involved in the survey distribution process, which eliminates any potential for airline employee-introduced bias and also frees cabin crews from an extra burden. Instead, GAP interviewers in each airport approach air travelers at their gate or in the departure lounge to engage them in the survey process, which entails completing a four-page questionnaire prior to departing and a second, shorter form at the conclusion of their flight.

“We interview them when they are the most available: before the flight,” says David Perroud, GAP manager, P. Robert and Partners. “Many people at that time are just waiting for the flight to board so anything that can distract them from waiting is welcome. We have very few refusals.”

The respondent gives the completed first part to the GAP representative, who then instructs him or her on how to return the second part, either by the postage-paid mailer, by fax or by telephone.

In 20-some questions, part one of the survey examines the performance of airline staff during reservation and check-in, the respondent’s impression of the airport facilities, and gathers other information about flying habits, frequent flyer program membership, and demographic data.

Part two looks at the boarding process, the airplane’s cabin comfort and features, the cabin crew, food and beverage service, onboard amenities, and post-flight impressions in addition to overall assessment of the airline and the flight in question.

Rather than contacting respondents after their flight by phone or mail, the GAP study reaches the passengers while the experience is still fresh in their minds. “We want to capture their opinions very close to the moment of truth,” says Paul Lai, general manager - marketing research, Delta Air Lines, Atlanta, a GAP subscriber.

The survey boasts a response rate of better than 50 percent, that is, about 50 percent of respondents who complete part one also follow through and submit part two.

“That response rate is a far cry from the less than 10 percent response rate from previous studies,” Lai says. “When the response rate is so low, results could be inconsistent and unreliable. We would see results going up or down without any reasonable explanation. That’s when internal customers may question the validity of your research, and rightly so.”

While Lai does not want to cite specific examples of issues uncovered by the research, he does offer that the GAP research gives Delta opportunities to benchmark industry leaders in any service area and then act accordingly. “If we see that airline X is doing very well in the area of food and beverage, for example, we may want to find out what is going on there. So then we put our product people on the case and have them do some competitive flying to find out what they are doing to make their rating so high.”

As is typical with syndicated studies, participants receive data on their own performance as well as that of competitors. “The first and most important thing with GAP is that it is a benchmark,” Perroud says. “Airlines used to have a lot of information about their own passengers. Now they can measure their own performance and compare it to the competitors.

“The other important thing is that it’s an ongoing survey, so they can track their passenger satisfaction over time. So if they change something in their in-flight service, the food perhaps, they can see how their rating is affected. They can track quarter-by-quarter.”

Study subscribers receive results quarterly, in the form of data tables, raw data in SPSS or Quanvert, etc., and an Excel macro that allows them to produce charts from the data. Every six months a full report is issued.

Active in development

The airlines have been active participants in the development of the GAP study. “The design process has been a long one, involving around 15 airlines,” says Perroud. “We have done two different pilot studies and also have analyzed the study to see what is the most effective way of asking the questions. All of these airlines were already asking those kinds of questions so we had a good starting point.”

Dissatisfaction with syndicated transatlantic route research brought various airlines together two years ago to discuss development of a new study, Lai says. “Since a lot of airlines felt that need, it was easy to get people together to talk about possible solutions. So taking the first step was not that bad.”

While the first step was easy, those that followed were considerably more difficult. The biggest problem was crafting a consensus among the participating airlines during the survey development process, Lai says. “Everything from how a question is worded to how we use the results, even the look of the graphics, has been a bit challenging. There are so many airlines involved, and each has its own quirky preferences. Now, with the survey in full bloom, it is almost a miracle that we were able to pull it off.”

Help from the airlines was also instrumental in translating/back-translating the questionnaires into the various languages. Cathay Pacific helped with the translation of the Chinese questionnaire, for example. No matter the language, the survey forms use the same scales and are structured the same way.

Airline involvement in the study is ongoing. For example, Lai and other airline representative members of the GAP Technical Committee met in London in January and in Miami in October to discuss a host of issues pertaining to the survey, from adding new routes to data weighting options.

Integral part

As the link between the survey and the respondent, the interviewers are an integral part of the process and great care is taken to train them, monitor their performance and provide them incentives for doing good work.

“When we started a year ago we had a response rate of 44 percent and now it’s up to 52 percent and every quarter it is improving. The interviewers play a key role and they are probably one of our big strengths,” Perroud says.

“It is very important that they are smartly-dressed and professional in appearance. And they have to be very easygoing with other people. They are given a script to follow as a guideline but we encourage them to sound as natural as possible and not merely recite the words.”

Perroud and other GAP representatives visit the airports to meet with the interviewers. In addition, PRP sends them a quarterly newsletter showing how they and their airport performed compared to others. “We do some contests on the response rates, not on how many surveys they distribute but how many completed surveys they get back. We rate the airports and the interviewers and the three best interviewers [globally] get a check.”

Service issues

Lai says there are two main service issues facing airlines today. One is operational reliability – arriving and departing on-time, not canceling or delaying flights. “We have found that that is one of the greatest drivers of satisfaction, especially for domestic service,” he says.

The second is the interaction between airline employees and passengers. “This is more a dissatisfier than a satisfier,” Lai says. “By that I mean people take for granted a basic level of civility from the staff, whether it’s the reservation agent or gate agent. If a representative has an attitude problem, and if a customer doesn’t feel special, isn’t given the basic courtesy of a smile or a sorry or a thank you, they can become angry.”

In Delta’s case, with 78,000 employees, Lai says the question is, how do you manage and motivate them to give great, consistent customer service day in and day out? That human element is not as immediately controllable as on-time performance or the quality of in-flight meals. Measuring and monitoring customer service is part of the job of marketing research.

Lai says the GAP study provides good information on how Delta’s SkyTeam partner airline Air France is doing. “That does not replace the need for doing alliance-specific research with Air France and other partners but [the GAP study] certainly gives us a good idea of how they are perceived by their customers.”

Competitive view

The information from the GAP study affords Delta a one-of-a-kind competitive view, Lai says. “You have to put your performance into perspective, into the context of the other airlines. You might look at your own results and say things are stable or improving compared to last year. But if you find that your slight improvement actually puts you a few notches below other carriers, that is quite a different story. So the information we get from the GAP study is unique and invaluable in that sense.”