A Brief Quiz
Experimental Effects Identification

Chapter 10:

Design of Experimental Research






I. The Notion of an
experiment: the study of the effects of variables manipulated by the researcher in a situation in which all other variables are controlled, completed for the purpose of establishing causal relationships
confounding: when variation from one source is mixed (or confused) with variation from another source so that it is impossible to know whether effects are due to the impact of either variable separately or some combination of them
--deception: an ethical
  problem created when
  research participants
  have been either (1)
  uninformed that an
  experiment was being
  conducted or (2)
  intentionally misled
  about the nature of the
  study in which they are
   A.  Questions and
         Hypotheses in
         --hypotheses in
           experiments are
           phrased to
           explore cause-
           require that
           variables be
           capable of
experimental independent variables:  independent variables that are manipulated by the researcher
   B. The Concept of
control: methods researchers use to remove or hold constant the effects of nuisance variables
methods of control:
elimination and removal: removing a nuisance variable from the experimental setting
holding constant: limiting the range of intervening variables so that they are equal across studies (by 1. limiting the population, 2. using subjects as their own controls, 3. Counterbalancing [rotating the sequence in which experimental treatments are introduced to subjects in an effort to control for extraneous variables, such as fatigue or cumulative learning effects])
matching: pairing subjects on some variable on which they share equal levels and then assigning them to experimental or control conditions
blocking: adding a nuisance variable into the design as another independent variable of interest
randomization: assigning subjects so that each event is equally likely to belong to any experimental or control condition
--respondents may be
  selected at random from
   the population; they also
   may be assigned at
   random to experimental
   or control conditions
statistical control: use of statistical tools such as analysis of covariance and partial correlation to hold a nuisance variable constant
additional sources of error:
halo effect: influences from strong
positive or negative impressions of a
source of communication that affect
other ratings that follow
placebo effect: an occurrence in which
subjects show change even though
there is no experimental treatment
John Henry effect: subjects' inconsistency with their normal activity because they try extra hard when participating in experiments
"do nothing" control groups: since experimental and control groups should share all things except the experimental variable, control groups that "do nothing" are apt to differ from the treatment groups in additional ways
II.  Experimental Validity
    and Invalidity
experimental invalidity: errors that prevent researchers from drawing unequivocal conclusions
    A. Internal Invalidity
         --if all sources of
           internal invalidity
           are not controlled,
           it is not possible
           to claim that any
           observed effects
           were caused by
           the independent
           variable in the
internal invalidity: the presence of contamination that prevents the experimenter from concluding that a study's experimental variable is   responsible for the observed effects
history: events not controlled by the researcher that occur during the
experiment between any pretest and posttest;
selection: sampling biases in selecting or assigning subjects to experimental or control conditions;
maturation: changes that naturally occur over time;
testing: alterations that occur when subjects are tested and made testwise or anxious in ways that affect them when they are given a second test;
instrumentation: changes in the use of measuring instruments from the pretest to the posttest, including changes in raters or interviewers who collect the data in different conditions;
statistical regression: shifts produced when subjects are selected because of very high or very low scores on some test and then changes on that measurement are tracked in the experiment;
experimental mortality: biases introduced when subjects differentially (nonrandomly) drop out of the experiment;
interaction of elements: effects created by the interaction of selection biases with differential levels of maturation, history, or any other source of variation
    B. External Invalidity external invalidity: the degree to which experimental results may not be generalized to other similar circumstances
interaction of testing and the experimental variable: (pretest sensitization) a defect created when the pretesting makes subjects either more or less sensitive to the experimental variable
interaction of selection and the experimental variable: effects created by sampling groups in such a way that they are not representative of the population since they are more or less sensitive to the experimental variable than other subsamples from the same population
reactive arrangements: elements in the experimental setting that make subjects react differentially to the experimental arrangements rather than to the experimental variable alone
multiple treatment interference: if subjects are exposed to repeated additional experimental treatments, they may react in ways that are not generalizable to subjects who are uncontaminated by such additional independent variables
III. Specific Experimental
    A. Notation for

O: an observation of the study's dependent variable
X: the experimental variable
R: randomization

     B. Pre-Experimental
          1. one shot case
one shot case study: an experimental treatment is introduced and researchers look at effects on some output (dependent) variable without benefit for a control group
          2. one group
one group pretest-posttest: a case study with an additional pretest so that subjects can serve as their own controls
          3. static group
static group comparisons: a design adding a control group, but the two groups are not known to be comparable
    C. True Experimental
        1. Pretest-Posttest
            Control Group
              avoid using
              "change scores"
              since (1) change
              scores may not
              that make them
              easy to interpret
              with standard
              statistical tools,
              (2) change
              scores have
              lower reliability
              than the original
pretest-posttest control group design: a design including pretesting and posttesting individuals in a randomly assigned experimental group and control group
        2.  Solomon Four
             Group Design
Solomon four group design: a design that adds control groups to examine pretesting effects directly
        3.  Posttest Only
             Control Group
posttest only control group design: a design that controls for pretest sensitization by deleting the pretest entirely
    D. Factorial Designs factorial designs: experimental designs that include more than one independent variable
         1.   Uses in Research factors: variables that are broken down into levels; a.k.a. "variable factors"
levels: categories of each factor
single subject experiments:
experiments in which experimental conditions are presented to or deleted from the same subjects over time
--limitations: (1) most applicable
   to questions of individual
   difference, rather than patterns
   across people; (2) limited
   generalizability prevents
   examining hypotheses of social
   significance to most
   communication researchers; (3)
   limits the sorts of statistical tools
   that might be used to help analyze
         2. Interpreting Factorial
             a. Main effects main effects: dependent variable effects from independent variables separately
             b. Interaction
                   are indicated
                   by lines that
                   are not
                   parallel to
                   each other
                   are graphed
                --unlike ordinal
                   when crossed
                   are found,
                   are forbidden
                   from interpreting
                   the main effects
                   for the variables
                   involved since such
                   would be misleading
IV.  Some Elements Found in
      Good Experiments
interaction effects: dependent variable  effects from independent variables taken together
ordinal interaction: if lines drawn in graphic displays of the effect are not parallel and do not cross each other, the pattern is called an ordinal interaction
disordinal interaction: patterns of graphic displays of the effect that occur when lines drawn for each independent variable cross each other (a.k.a. "crossed interaction")
      A. The Pilot Test pilot test: inquiries that usually involve small samples of people (sometimes as small as ten or twenty people) who take part in an experiment to determine any difficulties with experimental materials
      B. Manipulation Checks manipulation check: a researcher's measurement of a secondary variable to determine that an experimental variable actually operated in a study
quasi experiments: experimental work where random assignment and control are not possible
--time series designs: measurement
   of subjects across different times
--separate sample posttest designs:
  posttests completed from another
  group of subjects that are close but
  not assured to be equivalent to the
  experimental group
--counterbalanced designs: designs
   that introduce several different
   experimental treatments presented
   to subjects in different orders or
      C. Care in Interpretation
           1. Resisting the
               Tendency to Infer
                Long Term
                Effects from Short
                Term Experiments
           2. Searching for
           3.  Desirability of