A global enterprise

Twice a year, during the spring and fall, IBM conducts a tracking study in 14 languages in 27 countries throughout Europe, North and South America, and Asia. One of every six sites where an IBM S/390 Parallel Enterprise Server is in use is sampled. With the basic purpose of capturing trend data on enterprise server, or mainframe, computing, the study tracks information on a variety of topics, including perceptions of IBM and competitive products, product demand, future acquisition plans, preferred marketing channels and preferred product information sources.

Respondents are asked about the computing equipment they have installed, their future acquisition plans, and their impressions of the various vendors. "This study is the most extensive sampling of the view of large customers inside or outside IBM," says Ed Hogarty, manager of market research and business support, System 390 brand marketing, IBM, Poughkeepsie, N.Y. "We track how we’re doing on an ongoing basis, anything from sales and service to product capabilities, announcement effectiveness, and awareness and purchase intent. We keep it broad because the primary purpose is to track trends, not to get a deeper understanding of customer wants and needs."

The study helps IBM see how successful it is in penetrating key industries and also gives an idea of how IBM and competitive equipment is used at both large and small businesses. "The survey represents solid, quantitatively supported data points. We analyze it and then review it with the sales force in each of the countries and get their interpretations, and come back with a set of the top 10 issues that the field is wrestling with, to which we then respond," Hogarty says.

The respondents, both IBM customers and non-customers, have responsibility for making computer system acquisition decisions at their respective companies. For the most part, the interviews are conducted by telephone, but some respondents are also given the option to complete the survey via diskette. IBM is exploring the Internet as another option.

Interviews are conducted by Princeton, N.J.-based research firm RONIN Corp., at its international Call Centre in London and by another research supplier. RONIN conducts the European and Asian interviews using Results for Research, a RONIN-developed project management research tool and CATI package. The software simplifies the navigation of complex questionnaires through a database driven system, allowing interviewers to select items from extensive lists. For example, interviewers working on the IBM project are required to select a computer model from a list of 4,000 brands and models.

To make sure results are consistent and applicable across each geography, the questionnaires are kept identical, with country-specific responses displayed only for the countries where they apply. The CATI package allows for questions and responses to be filtered on a country basis, which simplifies the cleaning, coding, and analysis processes, says Elizabeth McNair, senior project manager, RONIN Corporation.

Accurate translations

With so many languages to work in, accurate translation is a major component of successfully conducting IBM’s tracking study. For consistency purposes, RONIN uses two translation firms, one for European languages and another for Asian languages. In addition, IBM personnel in the respective countries review the questionnaire to ensure proper translation of technical terms.

Large-scale global or international studies require a simple and straightforward translation process. Prior to starting the translation process, approval and finalization of the questionnaire are essential to avoid repeated translation, project delays and added costs. RONIN typically conducts an English language pilot test to identify what, if any, changes must be made to the questionnaire before translation occurs. Clients attend these tests and changes are made to the CATI program in real time, allowing for immediate testing of new question or response wording.

McNair offers these tips to keep in mind when conducting international research:

  • Thoroughly investigate potential sample sources to make sure they are able to provide a representative sample of the universe to be studied. Request documentation from each sample source about the demographics of the file, and verification steps performed to ensure proper coverage. A few questions to ask include: Does it cover all regions/cities of the country? Are all vendors represented? Are all size companies and industries included? What proportion of the universe is covered? Are there any biases in the companies which are covered or omitted? How is the sample source assembled? How is it updated?

  • Interview length varies by language and country and is typically longer than studies conducted in the U.S. For example, a 20-minute English language survey could take 18 minutes in Italian due to the speed of the spoken language. The same survey would take approximately 23 minutes in French due to respondents’ tendency to elaborate on their answers. Japanese interviews would be even longer.

  • Use native language interviewers to enhance the comfort level of respondents and to increase cooperation rates. RONIN has found that by using native language speakers, respondents from those countries are more willing to participate and also find it easier to converse. This is particularly important for qualitative open-ended questions, or questions such as awareness and brand image, where subtle nuances must be identified.

  • Use a single call center for a given geography to provide a single point of contact and consistency of results across countries and languages, with rapid turn-around. "We maintain maximum control over the project, always knowing precisely where it stands in all countries, and are able to expedite any changes or special instructions that are to be implemented worldwide. Briefings, training, and pilots can be conducted and monitored simultaneously," McNair says.

  • For analysis and reporting purposes, remember that a few select countries are not representative of Europe as a whole. In many cases, research is typically conducted in France, Germany, Italy, and the U.K. However, countries such as Spain, Sweden, Switzerland or Poland are typically left out of the mix. Consequently, the data should only be interpreted for the surveyed countries.

Quite cooperative

Hogarty says that because the survey, now in its fourth year, is widely viewed as the most comprehensive in the industry, respondents are usually quite cooperative about participating. They’re offered the incentive of a summary of the results, which gives them a chance to see how they compare to the rest of the world. "In order to keep the response rate up, we send a feedback report to those who participate. That’s helped out quite nicely. We also put a variation of that report on-line, for our executive customers. The goal is to get them interested in the results and get them willing to participate."

Still, it’s a struggle to get busy people to take time to complete the survey. "The feedback report helps, but it’s a constant challenge to keep the response rate up. We’re always looking for new ways to make it easier for the customer to take the survey. But we’ve found that most customers are interested in trying to help."

IBM article sidebar:

Enabling interviewers to quickly navigate through thousands of similar-sounding brands and models has been a long-sought "holy grail" for the research industry. Likewise, enabling interviewers to add new models to a choice list, without incurring the time and cost of post-interview coding, had historically hampered the process of keeping such product lists up-to-date. When conducting global research, these problems are greatly compounded by the mix of languages, remote geographies and country-specific needs.

IBM and RONIN overcame these difficulties through the use of the database-oriented "brand-list" question found in Results for Research, the CATI system used for IBM’s global research project.

By storing the model list as an external Results database, new models can be added to the list from within an interview. This way, new models can be instantly available to all other interviewers without requiring a separate coding step. Because the same database of models can be accessed from multiple-language questionnaires, models added in Germany, for example, can be instantly available to Spanish, Italian and Japanese interviewers.

It is also possible to identify specific attributes to each model (or record) in such a brand-list database. By adding performance-related information on each pre-coded model in the database, it was possible to base the questionnaire branching on the computing power of each model selected - ensuring IBM the detail it needed on its customers, while reducing overall interview length.