TQM comes to Main Street

Civic officials around the country are discovering that their "customers" - the residents of their communities - expect and deserve the same quality treatment that many businesses are striving to deliver. Even very satisfied residents may display complex requirements and expectations that must be understood and addressed. In addition to standard cross-tabulations and percentages, three techniques are especially useful in analyzing citizen survey research results: quadrant analysis, gap analysis and an examination of verbatim responses to open-ended questions. These techniques were used to analyze a citizen survey conducted for Lakewood, Ohio (pop. 59,000), a suburb bordering Cleveland.

"A customer service focus is a major premise in a total quality initiative," says Gale Fisk, Lakewood’s finance director. "We needed to know if the citizens of Lakewood, our customers, were pleased with service delivery. We needed to see areas where we were strong and areas that needed improvement. We also needed information on the importance of services and programs. As administrators, we felt more emphasis was needed on upkeep of the infrastructure, such as street reconstruction. However, the key question is, do the residents see this as a priority? The survey has provided this information, and it will be used to guide our planning for service levels and programs."

The research indicated that 88 percent of the residents were satisfied with life in the city. However, even with this high overall satisfaction score, the research pinpointed critical areas where city officials needed to plan and implement improvements.

Joint development

The questionnaire was developed jointly by the City of Lakewood’s total quality management team and National Survey Research Center (NSRC), a Cleveland marketing research and consulting firm. Mary Holloran, Lakewood’s director of planning and development, emphasized the importance of the joint development. "We relied on NSRC’s expertise, but at the same time, members of the total quality management team needed to provide essential input. Acting on the survey results was easier because all members participated in the development of the research questions."

The overall research objective was to capture information about the importance and satisfaction of 21 specific city services. Five-point rating scales were used. Importance was captured with a scale where a 5 indicated "very important" and a 1 denoted "not important at all." Satisfaction ranged from a 5 indicating "very satisfied" to a 1 denoting "very dissatisfied."

Telephone interviews were chosen rather than a survey by mail. Better response rates and a subsequent reduction in non-response bias were important factors in the choice. In addition, a critical component of the analysis methodology involved the analysis of verbatim comments to open-ended questions. Interviewer clarification and probing are absolutely essential to this analysis. Verbatim responses are often a weak point of mail surveys. Some researchers have found that the inclusion of open-ended questions also reduces the response rate to mail surveys.

Alan Dutka, president of NSRC, believes that obtaining opinions from a random selection of residents is an essential aspect of the research. "Public officials get plenty of feedback," he says, "but it usually comes from a very small segment of the population whose views may not be representative of the population as a whole. The citizen survey puts issues and viewpoints in a much broader and more accurate perspective."

Quadrant analysis

A quadrant analysis, where average satisfaction is plotted against average importance for each city service, was an essential first step in analyzing the survey results. The quadrant analysis for the Lakewood research is displayed in Figure 1.

Each of the four quadrants has a specific interpretation:

1. The upper right quadrant (high importance and high satisfaction) represents current city service strengths. These include emergency fire service, emergency medical service, police service, parks, recycling and refuse collection, snow removal and street lighting.

2. The lower right quadrant (high importance but relatively lower satisfaction) denotes services where satisfaction should be improved. The services in the category include fire safety education, public health services, water system, sewer system and routine street maintenance.

3. The lower left quadrant (relatively lower importance and relatively lower satisfaction) represents lower priority services. These are services for senior citizens, street cleaning, youth services, forestry program, child care services, animal control, city newsletters and communications, building code enforcement and sidewalk maintenance.

4. The upper left quadrant (relatively lower importance and high satisfaction) is often interpreted as representing "overkill" services where effort exceeds expectations. No service attributes are represented in this quadrant.

The five service attributes in quadrant two are candidates for immediate attention. Citizens placed a high importance on these attributes but also reported relatively lower satisfaction: fire safety education, public health services, water system, sewer system and routine street maintenance.

Gap analysis

Quadrant analysis is very useful in interpreting importance and satisfaction ratings. The technique, however, does not explicitly identify the gaps between importance and satisfaction. For example, a large gap could exist between importance and satisfaction even though a service attribute appeared in the "high importance and high satisfaction" quadrant. Consequently, gap analysis was the second component used in analyzing research results.

The gaps between average importance and average satisfaction for each city service are presented in Figure 2. A big gap between importance and satisfaction usually signifies a problem. The largest gap occurs with building code enforcement. Other relatively large gaps occur with routine street maintenance, water system, sewer system, and sidewalk maintenance.

The smallest gap occurred with city newsletters and communication. Three service attributes registered "negative" gaps where average satisfaction exceeded average importance: child care services, street cleaning and services for senior citizens. These may be "overkill" situations, and two of the three attributes (street cleaning and services for senior citizens) also appeared very close to overkill designation in the quadrant analysis.

Quadrant analysis and gap analysis offer two perspectives regarding the identification of service attributes that require immediate attention. Combining the results of the two analyses, seven attributes were identified as potential problems:

Quadrant Analysis Gap Analysis
Water system X X
Sewer system X X
Routine street maintenance X X
Fire safety education X
Public health services X
Building code enforcement X
Sidewalk maintenance X

The list of 21 original attributes was narrowed to seven potential problem areas. An examination of verbatim responses provided the final piece of information required to prioritize and develop corrective action programs.

Verbatim responses

Residents registering low satisfaction scores for service attributes were asked the specific reasons for the dissatisfaction. This is an essential component of the research because the reasons for dissatisfaction are often complex and not always obvious. Interviewers must thoroughly probe to discover why dissatisfaction exists, Dutka says. "Getting good verbatim information is the most difficult part of the research process. A response like ‘service is poor’ is not adequate enough to understand the cause of the dissatisfaction. You have to find out why the respondent thinks service is poor. Control of the quality of verbatim responses is a major reason for selecting telephone research."

Verbatim responses supplied the vital information necessary to properly interpret the causes of dissatisfaction with service attributes. Of special concern were the seven attributes that were identified as potential problems. For example, building code enforcement, the service attribute with the largest gap, was especially challenging. A substantial number of Lakewood homes are rental properties. Homeowners felt that renters do not properly maintain property and therefore building code enforcement should be increased. Renters agreed with the premise but asserted that absentee landlords are responsible for property maintenance. Consequently, both homeowners and renters supported building code enforcement. Another significant set of property owners, however, was dissatisfied for the opposite reason. They felt that the building codes are too stringently enforced and consequently discourage property improvements.

"Clearly an emphasis needs to be placed on rental units and residential code enforcement," says Lakewood Mayor David Harbarger. "We have an outstanding Building Department with sufficient personnel, but we need to shift priorities and be more responsive to residents’ concerns."

Similar in-depth analyses of verbatim responses associated with the other six service attributes of special concern indicate the following:

  • Three separate issues are involved in dissatisfaction with the water system: the taste of the water, the price and problems with maintenance of the system.
  • Basement flooding is the central issue explaining dissatisfaction with the sewer system.
  • Street repairs, considered necessary and important, were perceived as being neglected by the city.
  • The major complaint about fire safety education was that the residents do not know anything about the service. Many residents stated that they did not know it even existed.
  • Comments about public health services ranged from complaints about the services being provided to people who are not taxpayers of the city to complaints that the respondents were not receiving any of the benefits.
  • Residents complained that nothing was being done to replace unsafe sidewalks that are uneven and cracked.

The verbatim responses also provide valuable information about the service attributes that were not identified as problem areas. For example, dissatisfaction with police exists because the residents believe that police are targeting the wrong problems. They say that emphasis is inappropriately placed on traffic violations rather than more serious crimes. The residents, however, do not express fears about safety or violence.

There are many facets involved in dissatisfaction with recycling and refuse collection. Many residents were dissatisfied because other residents do not participate in the recycling program or are not forced to cooperate with the recycling program. Some residents were confused about what can be recycled while others were dissatisfied because the city will not collect certain items. Residents complained that they do not know how refuse is collected from apartments. A few thought that the refuse collectors made a mess of the area while other residents viewed the operation as too large in terms of people and vehicles.

Establishing priorities and allocating resources

The quadrant, gap and verbatim response analyses were combined with more detailed demographic information to establish priorities and allocate resources. For example, changing building code enforcement priorities was an immediate consequence. Increased publicity about the fire safety education program was an easy and effective method to improve satisfaction with this attribute. From a planning standpoint, the survey results will be used to bolster a recommendation for a street maintenance program in the administration’s 1996 budget.

The City of Lakewood has obtained the information needed to plan and prioritize programs and services. The perspectives of the community are accurately reflected since the survey respondents were selected in a random manner. In addition, benchmark information has been created to monitor progress over time. The importance of feedback from citizens will become even more critical as fiscal pressures on governments continue to mount.