Fax versus mail in gathering data: An analytical study

By David Liu, Nanda Ganesan | May 1, 1996

Reading time: 9 minutes

Abstract

Fax and mail surveys are popular research options when on a tight budget. California State University, Los Angeles's Information Systems Department used mail and fax surveys with libraries to compare these two research methods.

Research Topics:: Mail Surveys | Quantitative Research
Industry/Market Focus:: Higher Education
Content Type: Research Methodology

Share Print

Editor's note: David Liu, Ph.D., is associate professor of Information Systems at the School of Business and Economics, California State University, Los Angeles. Nanda Ganesan, Ph.D., is professor of Information Systems at the School of Business and Economics, California State University, Los Angeles.

With the increasing popularity of the fax machine and fax broadcasting - which allows users to send a document to hundreds of locations simultaneously - the fax survey has become a viable option for researchers looking for an inexpensive method of reaching respondents. A mail survey is also quite a popular choice for those who need to conduct research on a tight budget. One way to determine the relative effectiveness of these two approaches is to send the same information via the two different communication vehicles and analyze the resulting response rates.

This article presents results of a study conduced by the Information Systems Department at the California State University at Los Angeles. For the purpose of the study, a survey was sent via mail to one group of recipients and transmitted by fax to another group of recipients. The number of responses and the rate of response from each communication vehicle were tabulated. These results were analyzed to determine the effectiveness of postal delivery versus fax broadcasting.

The targeted recipients of the survey were a group of libraries known to have CD-ROMs. The survey focused on their utilization of fax and CD-ROMs. To better distinguish the responses to the faxed survey from those to the mailed one, the fax broadcasting was conducted a month in advance of the postal mailing. In addition, the fax survey had a distinctive marking on the bottom signifying that it was transmitted to the recipients via fax. However, in analyzing the responses, no consideration was given to the fact that the fax broadcast occurred during a different time period than the postal mailing. In other words, the analysis did not take into consideration calendar-related or chronological factors. For example, if more people were on vacation in March than April, fewer responses would have been received.

Respondents to both surveys had the option to return the completed forms via fax or mail. As the responses to the fax broadcast were received, the date of receipt was logged, as well as the responses to each one of the questions in the survey. For fax responses, the date of the fax transmittal was logged. For mailed responses, the postmark date was logged. The same method was used and the same data was gathered on the surveys that were mailed.

The surveys were mailed/faxed to 1,653 libraries with CD-ROMs. Altogether 1,300 surveys were transmitted by fax and 353 were mailed. The 1,300 faxed surveys resulted in 385 responses being received by fax and 178 responses being received by mail. The 353 surveys mailed resulted in 77 responses being received by fax and 63 responses being received by mail. These results are tabulated in Table 1.

It is important to view the results in light of the following environmental conditions.

Keep in mind that the recipients are libraries not businesses.
The unsolicited fax transmittal is a survey from a university conducting research as opposed to an unsolicited fax transmittal of an advertisement.

Because the above conditions apply to both the fax broadcasting and postal mailing, it is reasonable to assume that the effect of these conditions will cancel one another while comparing both modes of transmission.

Observations

The following observations are made based on the results shown in Table 1.

Overall response

Overall, the response rate for fax broadcasting is somewhat better than that for the postal delivery. (563/1300 > 140/353)

Mode of response

Faxed surveys resulted in a higher percentage of the responses being returned by fax when compared to the faxed responses received from mailed surveys. (385/563 > 77/140)
The mailed surveys resulted in a higher percentage of the responses being returned by mail compared to the responses received by mail from the faxed surveys.(63/140 > 178/563)
Overall, fax was the mode of choice for returning the surveys irrespective of the mode of transmission of the surveys. ((385+77)/(1300+353) > 0.5)

Overall response comparison

To test the validity of the observation that the response rate for the fax broadcasting was somewhat higher than that for postal delivery, a z-test was conducted. The test was based on the null hypothesis that there was no difference in the response rate between the two modes of transmission. The test produced a p-value of 0.2186, indicating that the differences were not statistically significant. Although the simple ratios appear to favor of the facsimile transmittals in terms of responses received, statistically there is no evidence to support this claim. Table 2 summarizes the result from the test.

Mode of response comparison

There are three parts to the observations made above on the mode of response. Because of the complementary nature of the first two observations, only a single test is required to test the validity of both observations. Therefore, the analysis is divided into two sections with the first section addressing observations 1 and 2 and the second section addressing observation 3.

Correlation between mode of transmission and mode of response

Altogether, 68.4 percent of the responses were faxed back from the fax broadcast, which was higher than the 55.9 percent faxed back from the mailed surveys. Likewise, 44.1 percent of the responses were mailed back from the mailed surveys, which was higher than the 31.6 percent responses received by mail from the fax broadcast. This led to the observation that a higher percentage of responses were received by fax transmission for faxed surveys compared to the percentage of faxed responses received for the mailed surveys. It also led to the observation that a higher percentage of responses were received by mail for mailed surveys compared to the percentage of mailed responses received for the surveys sent by mail. As mentioned earlier, because of the complementary nature of the ratios, only one z-test was required to test the validity of both observations.

The test was conducted based on the null hypothesis that the percentage of responses received by fax was the same for both the faxed and the mailed surveys. The test was conducted for only the responses received by fax because the same p-value will apply for the responses received by mail. The test produced a p-value of 0.0028 resulting in the rejection of the null hypothesis. This can be interpreted as a confirmation of the observation that a higher percentage of the responses are likely to be transmitted by fax for faxed surveys and by mail for mailed surveys. It is important to note that the comparison of responses received by fax is being made between the surveys sent by fax and those sent by mail. The results are summarized in Table 3.

Fax as the mode of response

The other observation made under the Mode of Response in the "Observations" section above is that the overall percentage of responses received by fax is higher than the responses received by mail irrespective of the initial mode of transmission. In other words, the observation holds true for both the fax broadcasting and the mailing of the surveys. In order to test the validity of this observation, a z-test was conducted based on the null hypothesis that the overall percentage of responses received by fax is 50 percent. In other words, the null hypothesis states that the probability of receiving the response by fax is 0.5 irrespective of the mode of transmission. The test produced a p-value of less than 0.0004 indicating that there was a significant difference in the manner in which the recipients responded. It means that the responses are more likely to be faxed back irrespective of the mode of transmission of the surveys to the recipients. This is also an indication that fax is increasingly replacing mailing as the mode of transmission in the organizations surveyed. Table 4 summarizes the test result.

Response time analysis

In addition to analyzing the manner in which the recipients responded to the survey, the study also investigated the response in terms of timeliness. For this purpose, the number of responses were tabulated for the time intervals of five, 10, 15 and 20 days. Table 5 shows the tabulated results for the surveys sent by fax and Table 6 the results for the surveys sent by mail.

The data in Table 5 indicates that within the first five days of the fax broadcast, 267 responses were received by facsimile transmittal and 129 responses by mail. The 267 responses represent 69.3 percent of the total responses received by facsimile transmittal and the 129 responses represent 72.5 percent of the total responses received by mail. Over 90 percent of the overall responses from each communication vehicle was received within 10 days of the initial transmission of the surveys, and nearly 98 percent of the overall responses were received within 20 days.

The data in Table 6 indicates that for the mailed surveys 49.3 percent of the total number of faxed-back responses were received within the first five days. In the same period, 36.5 percent of the mailed responses were received. Also, within the first 10 days, 77.9 percent of the faxed-back responses and 73.0 percent of the mailed-in responses were received. Based on the data presented in Tables 5 and 6, the following observations are made with regard to the response rates for the different time frames for the two modes of transmission.

Responses received in five to 10 days: The response rate from fax broadcast is better than the response rate from postal delivery during the first five to 10 days.

Responses received in 15 days: After 15 days, however, the rates of response for the fax broadcast and postal delivery begin to converge to 90 percent.

First, two z-tests were conducted, one for the returns received within the first five days and the other for the returns received within the first 10 days, to investigate the validity of the first observation. In each case, the null hypothesis stated that there was no difference in the response rate between the two modes of transmission of the surveys for both time frames. The tests produced a p-value of less than 0.0004 for the five-day period and a p-value of 0.001 for the 10-day period. Table 7 contains the test results. The p-values are statistically significant to support the observation that the faxed surveys commanded a better response rate within the first five to 10 days. The results seem to confirm the general notion that there is a sense of urgency attached to a facsimile transmission and that most people give a higher priority to responding to faxed information. Another contributing factor is the elimination of the time delay that is usually associated with the mail delivery.

Another z-test was conducted to compare the 15-day response rate between the faxed and mailed surveys. The test was based on the null hypothesis that there was no difference in the percentage of responses received between the two modes of transmission for the time period concerned. A p-value of 0.067 obtained from the statistical tests supports the preceding null hypothesis. This p-value is also included in Table 7 so that each p-value can be viewed in its own perspective and in comparison to other p-values.

Conclusion

In drawing a conclusion, there may be a tendency to view the results in light of the fact that the surveys were sent to libraries as opposed to customers or typical business organizations. But, because the study is comparative in nature and it evaluates responsiveness, the results yield to interpretations that are somewhat free of impact from environmental conditions. If the study had focused on quantifying the results, such as establishing a response rate for the faxed surveys, the environmental conditions may then have to be taken into consideration. As such, the results that are summarized here can be applied to most practical situations.

Individual mode of response. Faxed surveys resulted in a higher percentage of the responses being returned by fax when compared to the percentage of faxed responses received from mailed surveys. Likewise, the mailing of the surveys resulted in a higher percentage of the responses being returned by mail when compared to the percentage of responses received by mail from the faxed surveys.
Mode of response. Overall, fax was the mode of choice for returning the surveys irrespective of the mode of transmission of the surveys.
Timeliness in response. Fax broadcasting produced significantly better results within the first five to 10 days in terms of responses. However, the responses received within the first 15-day period were the same for both modes of transmission.

Our results partly corroborate the findings made in a related 1992 study.1

References

1 Dickson, John P.; and MacLachlan, Douglas, L. "Fax Versus Mail Survey." Marketing Research: Management and Applications. Volume 4, Number 3. September, 1992. p. 26.