Teeing off with new technology

For golf schools, the advertising brochure is a critical marketing tool. Selection of a school is largely based on the text contained in these brochures (which are mailed to prospective students who respond to ads in major golf magazines) since students rarely visit a school in advance and word-of-mouth advertising is not prevalent. Sales are gained or lost depending on the messages conveyed in the brochures.

The Ben Sutton Golf School, Sun City Center, Fla., has a unique advantage over its competitors: all instruction is conducted under actual playing conditions on a real golf course - not a driving range or practice green.

The task of communicating this major competitive advantage in the text of the school's new brochure fell to The Becker Group, a Canton, Ohio, ad agency. To do this, Glen Becker, president of the Becker Group, needed an objective critique of the new advertising copy. He turned to artificial neural network analysis.

Complex patterns

Artificial neural networks identify complex patterns of information in text data and then determine the association among the different patterns. Thus, the technique is well-suited to analyzing several copy alternatives to determine which options best convey the intended message. Using neural network analysis, subtleties and potential trouble spots in advertising copy are detected early in the creative process.

Alan Dutka, president of the National Survey Research Center, a Cleveland firm that performs customer satisfaction and other research for the Ben Sutton Golf School, says that "neural networks are appropriate when the research objectives are to determine the major ideas expressed in a text and to study the relationships among the words and phrases."

An artificial neural network analysis generates quantitative information in three distinct phases:

  • word counts identify the most frequently used words;
  • cluster analysis determines how words are grouped together to form major themes or ideas;
  • connection strengths among words reveal the interrelationships among all the words.

These three phases were used to evaluate existing and new advertising brochures for the Sutton Golf School and also the brochures for eight major competitors. Figure 1 contains the words most frequently used in the Sutton school's current brochure and the brochures of two competitors with dramatically different advertising messages.

The contrasts among the schools are apparent: The Original Golf School presents a very personalized message emphasizing words including "you'll," "golfer" and "student." The Professional Golf School targets the resort aspects of the learning environment. Among the frequently appearing words are "Mountain Valley," "resort" and "tennis." The message of the existing Sutton brochure is less obvious. "Green" and "fees" relate to the fact that green fees are included in the tuition cost. "Day" is used frequently to promote the flexibility of three- , five- and eight-day classes.

A cluster is worth a thousand words

Though they provide valuable insight, word counts don't show how the words are grouped together to develop concepts. The next analysis phase quantifies the relationships among the words. These analyses are similar to statistical procedures used to examine numerical data.

The mathematical connections among words are strengthened when they appear close to each other in the text and are weakened when they are far apart. Consequently, connections that are frequently reinforced will grow strong while infrequently reinforced connections generate weaker (and possibly even negative) connections.

Negatively connected words inhibit each other, so that activating one word actually deactivates the other, Dutka says. Negatively connected words may signify an impending disaster. For example, an advertising strategy based on attributes not commonly associated with a product might work in reverse and stimulate the sales of a competing product.

These mathematical connections are used as input to a statistical cluster analysis. Figure 2 summarizes the three most significant clusters in the Ben Sutton, Original and Professional golf schools' advertisements. Both the Sutton and Original Golf School brochures generate clusters associated with choices in class length. One of the major clusters for the Original Golf School is very direct: "You'll learn." The resort aspects form the major clusters for the Professional Golf School.

Applying cluster analysis to text data provides valuable information regarding how strongly words are related to each other. But artificial neural networks don't stop there.

Words activate words

One facet of neural network analysis is investigating how combinations of words activate other words in the text. Using an interactive neural net computer program, an analyst can select words or phrases from the text and determine what other words in the text are activated. Words are analyzed, not as they exist in one arbitrarily selected cluster, but as connections to all other words in the text. Knowledge and experience are required to properly interpret this research phase, just as expertise is required to properly interpret focus group transcripts or a multiple regression analysis. The results can be very revealing.

Figure 3 summarizes the words activated by the names of the respective golf schools. Once again, the Original Golf School's message is very clear: "You'll learn the game." Ben Sutton's name activates words relating to free greens fees and days of instruction. The critical message of learning under actual playing conditions is not evident.

Though the neural network analysis indicated that the advertising copy needed improving, Becker's strategy was to fine-tune the original message rather than initiate radical changes. "The brochure had a strong track record of attracting business over a long period of time," says Becker. "We knew it was good but we wanted to make it excellent."

As Becker revised the copy, Dutka performed the neural network analyses. The goal was to group words more effectively while using essentially the same words in about the same frequencies. This would ensure an improvement while not drastically changing the message.

Figure 4 presents a summary of the major clusters in the original and revised brochures. The intended message "under actual playing conditions" is the second most significant cluster. "Real instruction" and "personalized lessons" highlight the quality of the school. "Stop action video" is more dramatic than a former cluster composed of "video analysis." "Graph check" refers to a unique and creative video methodology used to analyze a student's swing while a grid is superimposed over an image of the student.

The "net" result

Becker views the artificial neural network analysis as a valuable safety-check in the agency's copy design efforts. "The computer analysis removes the biases that are always present when we analyze our own work. In the case of the Ben Sutton Golf School, the artificial neural network analysis indicated that the major intended messages are effectively communicated in an exciting manner."

Dutka predicts growing acceptance of using artificial neural networks in advertising research. "When neural networks are properly analyzed, later research is improved," he says. For example, diagnostic testing in an early preproduction state will enhance the effectiveness of second stage focus group or mall intercept research.

"Many problems will be rectified by the initial neural network analyses. This will allow more time for in-depth research with a greater opportunity to investigate the subtleties that contribute to effective advertising," Dutka says.