Up A Quiz for Chapter 9 Questionnaire Problems Survey Studies

Chapter 9:
  Design of Descriptive Empirical Research
 in Communication




I. Invitations to Empirical Research
   --problem statements and hypotheses that focus on current
     descriptions and relationships (attempts to: ascertain norms;
     establish goals; develop methods)

  --data that are quantitative; research questions often addressed
    by a combination of methods to "triangulate" research methods to
    draw upon all available information

empirical: means observable; most people use the term to refer to "scientific research"
descriptive empirical research: survey research in which contemporaneous data are gathered to answer research questions dealing with ascertaining norms, establishing goals, or developing methods.

experiment: a type of study using experimental methods in which researchers examine the effects of variables manipulated by the researcher in situations where all other influences are held constant. Variables are manipulated or introduced by experimenters for the purpose of establishing causal relationships.

experiment: a study that introduces a variable and determines its effects while controlling all other variables

II.  Survey Research

survey: the process of looking at something in its entirety
survey research: empirical study that uses questionnaires or interviews to discover descriptive characteristics of phenomena-

forms of descriptive studies:
--studies of behavior: facts and

opinions (studies designed to
determine the current status of conditions or attitudes)

--studies of status and

development (studies that trace the genesis and change of a
group of people throughout a life cycle

--methodological studies (studies

that deal with the development and validation of new tools and measuring instruments


III. The Questionnaire Survey
     A. Selection of Questions and
          Providing Instructions
          1.   ways to develop survey
                items: use standard forms in

questionnaires: surveys in which individuals are asked to respond to items they have read

resource works; select measurement forms that have been used in research pieces you have read; develop your own measures

2.      criteria for questions: to what

extent might a question influence respondents to show themselves in a good light? to what extent might a question influence respondents to be unduly helpful by attempting to anticipate what researchers want to hear or find out? to what extent might a question ask for information about respondents that they are not certain, and perhaps not likely, to know about themselves?


                --choices in question forms

direct (that ask for obvious reports) or indirect questions (that ask respondents to react in ways that imply information);
specific (that try to focus attention on individual activities) or general questions (that ask respondents questions) vs. statements (declarations to which subjects indicate how much they agree or disagree)

--types of questions








open ended questions: questions to which people respond in their own words
closed ended questions: questions to which people respond in fixed categories of answers
paired comparison questions: questions that ask respondents to make a judgment between alternatives taken two at a time
contingency questions: questions asked only of some respondents, determined by their responses to other questions
ranking questions: closed ended questions that ask respondents to rank order a set of options
inventory questions: closed ended questions that ask respondents to list all reactions that apply to them
matrix questions: closed ended questions that ask respondents to use the same categories to supply information
multiple choice questions: closed ended questions that ask respondents to select a category response from a range of possible responses

 --problems in question design:

double-barreled questions, loaded language, improper grammar, incompleteness, vagueness, ambiguous terms, lengthy items, complex questions, averaging or reconstructive questions, leading questions, abbreviations, imprecise questions, misspelling, awkward construction, items with only one logical answer, presumptive questions, elevated vocabulary, imprecise agents of action

3.      advice for phrasing items

a.   use consistent scales

b.   use consistent wording

c.   make sure the items can be used effectively by respondents (a pilot study is probably the best way to assure that new items created by the research can be used in meaningful ways by study participants

B.  Developing Instructions and Securing Informed Consent

1.      Securing Informed Consent:

--For anonymous surveys, the minimum to obtain informed consent includes that participants be informed about: (1) the purpose of the research, expected durations, and procedures; (2) their right to decline to participate and to withdraw from the research once participation has begun; (3) the foreseeable consequences of decline or withdrawing; (4) reasonably foreseeable factors that may be expected to influence their willingness to participate such as potential risks, discomfort, or adverse effects; (5) any prospective research benefits; (6) limits of confidentiality; (7) incentives for participation; and (8) whom to contact for questions about the research and research participants’ rights.

--If the responses are not to anonymous surveys or data collections, signed informed consent must be secured.

--Informed consent is not required only: (1) where research would not reasonably be assumed to create distress or harm and involves (a) study of normal education practices, curricula, or classroom management methods conducted in education settings; (b) only anonymous questionnaires, naturalistic observations or archival research for which disclosure of responses would not place participants at risk of criminal or civil liability or damage their financial standing, employability, or reputation, and confidentiality is protected; or (c) the study of factors related to job or organization effectiveness conducted in organization settings in which there is no risk to participants’ employability, and confidentiality is protected, or (2) where otherwise permitted by law or federal or institutional regulations.

--If study participants are not of an age that permits them to make such a decision for themselves, one of the parents of these minors must be contacted to secure informed consent on behalf of their children.

--In observations of overt behavior, researchers must “obtain informed consent from research participants prior to recording their voices or images for data collection unless (1) the research consists solely of naturalistic observations in public places, and it is not anticipated that the recording will be used in a manner that could cause personal identification or harm, or (2) the research design includes deception, and consent for the use of the recording is obtained during debriefing.

2.      Providing Instructions about Completing Materials

--Researchers also notice that if they introduce different sorts of questions, it is helpful to make sure that instructions for such materials are understandable to the people who eventually will complete such surveys.

3.   Providing Debriefing

--At minimum, the debriefing should include:

♦ information (if not previously revealed on the informed consent form) about the purpose of the study, including the general nature of the research question;

♦ information about an address where participants can get copies of the research report (either a physical address or a website);

♦ a contact person who may answer any questions about the project; and

♦ additional thanks for participating in the study.

      B.  Formatting

--order of questions

--grouping of items

--controversy about the number of scale points:

·  The number of scale positions that should be used remains controversial. (One study found reliability of Likert scales did not increase after five points [McDonald, 2004]).

--Whether to include a midpoint scale position remains hotly debated. On one level, reliability sometimes has been enhanced by adding a midpoint. On the other hand the midpoint position seems to be “overused,” a fact that reduces observed scale reliability.

--Aside from information about the age of participants (which must be indicated on the first page of questionnaires to assure that the respondent is qualified to give informed consent), in survey research (as opposed to interview studies) demographic information usually should be collected at the end of the questionnaire.

--Guidelines for length of questionnaires (limit instruments to six to eight pages; precode response categories by assigning a number to each possible answer for the respondent to circle; space the categories so that it is easy to circle one response without touching an adjoining one; provide simple instructions of no more than two sentences describing how to answer questions; use a different typeface for questions, response categories, and transitions or section headings; whenever possible, use arrows to indicate “skip” instructions.)


     C.  Determining Reliability and Validity

validity: the consistency of a measure with a criterion, which reveals the degree to which a measure actually measures what is claimed (to show that a set of questions is valid, one first must show that it is reliable)
reliability: the consistency of a measure with itself

          --additional controls:
            1. check questions

check question
: asking the same question twice at different locations in the questionnaire, usually once positively worded and once negatively worded

            2. measures of test taking behavior

social desirability: a measure of the degree to which people attempt to describe themselves in ways that they think are acceptable, desirable, or approved by others
MMPI Lie Scale: a scale to identify respondents who are attempting to avoid being candid and honest in their responses
Infrequency Index: a measure of the inconsistency of response

3.      polarity rotation

--With semantic differential-type scales it is a simple matter to rotate the poles so that all the positively oriented adjectives do not appear on the right or the left sides of the scale.

--When measures involve statements to which survey participants must indicate degrees of agreement or
disagreement, however, changing some positively worded items into negative worded items can be problematic. Negatively phrased items often reduce the reliability of the measure, especially for some groups of participants, and the use of some statistics can be jeopardized by the use of such items.

If researchers wish to add negatively worded items to prevent response set, they should not score the negative items as part of the final measures used in the study. Instead, the negative items may be used as control checks.           

polarity rotation: avoiding phrasing all items positively and avoiding placing all positive adjectives on the same side of the measurement items
--response set: a tendency for subjects to
   follow predictable patterns of
   responding to test items

     D.  Sampling Participants
     E.  Administering the Questionnaire

          --sound questionnaire studies include
             a section for participants to write
             open-ended comments in reaction to
             the survey materials


     E.  Analyzing and Interpreting Results

--survey research reveals relationships, but not relationships, but not causal relationships (experiments or long-term historical studies are required to make those sorts of claims)


IV. The Interview Survey
      --reasons for interviews:
        1. helpful when permission to
            participate is required;
        2. desired to reduce refusal rate;
        3. useful when it is important to
            record information (such as a
            respondent's manner and nonverbal
            actions) that might be lost by using


    A. Selection of Questions
         --types of interviews: structured
            interviews and unstructured


structured interviews: use of specific lists of questions
unstructured interviews: use of items that permit respondents to indicate their reactions to general issues without guidance from highly detailed questions

         --strategies for initial questions:

funnel questions: starting with an open-ended question and following up with increasingly narrow questions
inverted funnel questions: starting with a very specific question and expanding by   asking increasingly general questions

 --follow-up question strategies:

mirror questions: questions that repeat   previous responses to elicit additional   information
probing questions: directly asking for   elaboration and explanation
climate questions: asking respondents to explain how they feel about the interview climate and their willingness to answer questions

 --recommendations for interviewers:

1.  questions should be brief and to the point;

2.  words and phrases should be simple and in common day-to-day use among all groups in the               community;

           3.  questions should not include words
                that have strong emotional content;

           4.  questions must avoid all possible
                bias or suggestion in favor of or
                against a particular point of view;

           5.  questions should include all the
                important alternatives that may
                emerge on a given issue;
           6.  when the individual is asked to
                choose between different
                alternatives, this choice of
                alternatives must be given as early
                in the interview as possible.


             --a question of ethics: confidentiality
                and anonymity

confidentiality:  protection of the identity of research participants

anonymity:  protection of research participants be separating specific identities from the information given

B.  Training and Controlling Behavior of Interviewers

--Part of the training involves the use
  of “scripts,” which are, more or less,
   the “line” to be spoken during the
   interview process.

--Scripts of unstructured interviews
include only a few main issues or
questions that may start a conversation along a certain path. Yet, unstructured interviews secure
most information through the use of
follow-up or probing questions.

      C.  Determining Reliability and Validity    

      D.  Sampling Subjects

E.  Analyzing and Interpreting Results



scripts: in interview studies, the instructions given the interviewers regarding what they will say to respondents and how they will answer questions and probe for further information

          --the use of probes allows the
            interviewer to get information beyond
            the surface

          --rules for probing:
               1.  Repeat the question if the
additional detail,
                  elaboration, or misunderstood or
                  misinterpreted the question.

2.  Repeat the answer if the response is too vague to answer the question.

3.  Pause to indicate to the respondent that you need more or better information.

4.  Use neutral questions or statements to encourage a respondent to elaborate on an inadequate response.

5.   Use clarification probes when the
response is unclear, ambiguous, or contradictory.

            --credibility of questionnaire and

interview data may not be trusted if:  (1) respondents may not be aware enough to give useful responses; (2) recall is not known to be accurate

probe: in interviews, the use of scripted or spontaneous follow-up questions designed to help interviewers secure additional detail, elaboration, or clarification.

V.  Network Analysis

network analysis:  a set of research procedures for identifying structures in social systems based on the relations among the system's components rather than the attributes of individual cases

A.  Traditional Network Analysis

traditional network analysis:  a method
that obtains individuals' reports of their communication activities with others for the purpose of observing and describing the flow of information in a particular organizational system

1.     The Traditional Approach of  Network Analysis

a.  to construct a map of the

     interaction among people in an

b.  to diagnose problems in communication flow


c.   to identify the roles played by different group members communication

2.      The Method of Network Analysis

--general steps:

a.  selecting communication variables of interest;

b.  securing individual reports from all members of the group

grapevines:  informal flows of communication


c.  constructing network maps and tracking information flow


reciprocal pattern: one in which individuals share nearly an identical network pattern among themselves

liaison:  a person who links people of different networks together

isolate:  an individual who is not actively involved in any established network

d.  interpreting results by comparing the network to a desired standard

B.  Network Analysis Beyond the Traditional

--network analysis of social units or nodes can be people, departments, organizations, or message elements, such as words


1.   Semantic networks

semantic networks:  examinations of the relationships among words in a message

2.   Networks among Groups

networks among groups:  examinations of relationships among groups in an organization

3.   Networks among Organizations

networks among organizations:  examinations of relationships among different organizations

4.   Networks among Nations

networks among groups:  examinations of relationships among nations

C.    Limitations of the Approach

            1.   emphasizes structural information rather than content

2       does not explain the reasons for the relationships.