Cell Phone Sampling Summit II Statements on

“Accounting for Cell Phones in

 Telephone Survey Research in the U.S.”

 Prepared by Paul J. Lavrakas and Charles D. Shuttles

In February 2005, a meeting of statistical sampling and methodological research experts from the private, academic, and governmental sectors convened in New York City.  The purpose of this meeting was to facilitate the discourse within the telephone survey industry on the treatment of cell phones in Random Digit Dial (RDD) surveys of the general public in the United States. 

As a result of this meeting, it was proposed that several “advisory” statements be drafted for the research community concerning reaching cell phones while conducting telephone surveys.  The statements address: (1) the overall issue of accounting for cell phones in survey research sampling, (2) safety concerns of conducting research with a respondent while they are on a cell phone, (3) data quality concerns, and (4) the ethics of conducting research interviews with a respondent who is using a cell phone.

What follows are statements that reflect the general consensus of the Summit attendees about cell phone-related issues that should be considered by anyone conducting a RDD survey of the general population in the U.S.  As experience grows with conducting research interviews with people on their cell phones, these statements will need to be adapted to take into account that new knowledge.

Statement 1: Overall Statement on Considerations for Conducting Telephone Surveys with Respondents on Their Cell Phones
 

The impact of the increasing penetration of cell phones upon the Total Telephone Frame (TTF) in the United States is a legitimate concern to the survey research industry; especially as it applies to reaching housing units that are “cell phone only” (without a so-called landline). 

Cell phones traditionally have been excluded, whenever possible, from most RDD studies because they pose numerous challenges, such as: (a) determining sample design weights, (b) defining new response codes and response rate formulas (cf. Callegaro et al, 2005), (c) defining new calling and interview protocols, (d) re-evaluating the necessity for compensation for respondents, (e) working with cell phone providers to obtain 800 numbers that are truly free for cell phone calls and can be used by survey companies when asking the respondents to call back, (f) complying with the Telephone Consumer Protection Act (TCPA) restrictions, and (g) foregoing the geographic precision of landline telephone numbers. 

Currently, national estimates for the number of U.S. households with cell phones is approximately 70% (Piekarski, 2005), with cell phone-only households estimated to be upwards of 7% of all U.S. households and growing annually (cf. Blumberg, Luke, and Cynamon, 2005; Keeter, 2005; Piekarski, 2005; and Tucker, Meekins, Brick, and Morganstein, 2005).    Some research has shown that the exclusion of cell phone households does not bias survey results when poststratification includes weighting by age, e.g., with health survey estimates and with political polls (Blumberg, Luke, and Cynamon, in press; and Keeter, 2005).  It behooves the research industry to explicitly consider and clearly state/disclose how they are treating cell phones in their telephone surveys of the public. Furthermore, researchers need to continue to conduct empirical investigations of this issue on an on-going basis.

The decision to exclude or include cell phones in a sampling frame often will have major impacts upon a telephone survey.  At the simplest level, the total U.S. telephone frame can be partitioned into three components: (a) land-line telephone exchanges, (b) cellular telephone exchanges, and (c) mixed-use telephone exchanges.  It is important to understand that cellular telephone numbers are located in all three components of the frame; including the so-called “zero banks[1]” that traditionally have been excluded from list-assisted RDD samples.

Also, a researcher should account for many other cell phone-related issues, including but not limited to:

• Ported numbers from a landline exchange to a cell phone

• Landline numbers that are forwarded to a cell phone number

• The geographic implications of reaching a cell phone user in light of the target population that the survey is meant to represent, including whether any geographic screening of those reached on their cell phone is necessary;

• Cell phones can be for personal usage, business usage, or a mixture of personal and business usage. Thus whether these varied potential uses were taken into account when determining the eligibility of a respondent (e.g., the random selection of an adult from a sample household, or the identification of the person in the household most knowledgeable about a topic) should be disclosed;

• Cell phone ownership among teens and pre-teens is on the rise. Thus it is more likely to reach a ineligible person when doing RDD cell phone surveys of adults, and researchers should disclose how age eligibility of respondents was determined and assured (Kornblum, 2005); and

• Weighting for unequal probability of selection, including whether the cell phone is a personal device reaching only one potential respondent or a household device reaching more than one potential respondent.

Additionally, it is recommended that each telephone survey researcher should explicitly disclose how their survey is treating respondents who are reached on a cell phone. This includes, but is not limited to,

• Disclosure of how cell phone numbers are handled in sampling;

• How it was determined if a respondent was reached on a cell phone;

• At least a brief description of the procedures used to comply with the TCPA-related regulations regarding contacting  someone on a cell-phone

• Whether any inducements to stimulate cooperation (e.g., incentives) were provided to those reached on a cell phone

• Response outcomes and response rate(s) for those reached on a cell phone;

• Safety precautions that were taken to determine that the cell phone respondent was not put in harm by being interviewed; and

• Whether (and how) any weighting was done to adjust for those sampled from a cell phone.

As progress is made in further understanding how respondents reached via cell phones are best accommodated in telephone surveys of the general U.S. population, these suggested considerations concerning surveying respondents reached on their cell phones need to be continually updated.

Statement 2: Safety Concerns When Reaching a Respondent on a Cell Phone

The mobile nature of cell phone technology allows for a respondent to be engaged in numerous activities and to be physically present in various locations that would not normally be expected in reaching someone on a fixed landline number.  In particular, the operation of a motor vehicle or any type of potentially harmful machinery by a respondent during a research interview presents a potential hazard to the respondent and to anyone else in the general vicinity of the respondent (e.g., fellow passengers in the car). 

As such, any researcher who conducts a survey that includes respondents being interviewed on a cell phone should take appropriate measures to protect the safety of the respondent and those around the respondent. 

Statement 3: Data Quality Concerns When Reaching a Respondent on a Cell Phone

Many users of cell phones appear very willing to talk in all kinds of locations, including public and semi-private places, in which they are seemingly oblivious of those around them.  Nevertheless, a survey respondent reached on a cell phone may consciously or unconsciously limit the candor/openness, and thus the accuracy, of her/his responses depending on the sensitivity of the research questions (e.g., health and other sensitive topics; income, age, and other  demographic data; etc.). 

As such, whenever it is appropriate and based on the nature of the topics being surveyed, the researcher should determine whether the respondent on a cell phone is in an environment that is conducive to providing full and accurate answers to the questions the interviewer is asking.

Statement 4: Ethical Concerns and Good Business Practices When Reaching a Respondent on a Cell Phone

• Each researcher should anticipate the range of ethical and other business considerations associated with conducting a research interview with a respondent on a cell phone.  These considerations include, but are not limited to the following:

• Because of the cost structure of cell phone billing currently in the United States, there may be a financial burden upon the respondent for an incoming research call – one that does not occur with a landline phone.  Therefore, when appropriate, survey respondents reached on their cell phone should be properly reimbursed for their time on a research call.

• “Do No Harm” – survey researchers must proactively guard against putting anyone’s safety in jeopardy when contacting respondents on a cell phone. (see Statement 2)

• “Leave the Respondent With a Good Experience” – because people often are under special time constraints when speaking on their cell phones, survey researchers should take this explicitly into account whenever planning a questionnaire that may be used to interview someone on a cell phone. Thus, researchers should consider explicitly whether the length of an interview that is conducted on a cell phone should be shorter in duration than one conducted on a landline.

Bibliography