Reading, Understanding & Evaluating Research Glossary of research terms This glossary is presented to assist advocates in understanding commonly used terms when reading, understanding and evaluating research. This glossary drew from several sources, which are listed at the end of this document. These resources can be helpful to advocates who wish to learn more about understanding and evaluating research. ANONYMITY Research participants cannot be identified by any means or any person CAUSATION The act of causing; the means by which an effect is produced CORRELATION A measure of the relationship between two random variables, indicating the strength and direction of the relationship. For example, correlation is positive when two variables move in the same direction and negative or inverse when they move in opposite directions. *Note: Correlation DOES NOT EQUAL causation *Example: Binge drinking is often correlated with sexual assault in teenage girls. (“Sexual assault doesn’t cause,” 2011). However, binge drinking does not CAUSE a sexual assault. The sexual assault is caused by a person who violates another person in a sexual manner without that individuals’ consent. Sexual assault then can occur whether or not binge drinking is also taking place as it is an independent act. CONFIDENTIALITY Potentially sensitive or private information is being supplied with the understanding that the research participants’ identity, which may be known to the researcher, will be protected. DEPENDENT VARIABLE The variable that is affected by the intervention or independent variable; the one that changes; the presumed effect. EPISTEMOLOGY Concerns knowledge construction; asks what constitutes knowledge and how knowledge is validated. EXPERIMENTAL DESIGNS Designs in which subjects are randomly assigned to either the control or experimental group. Examples include: *Classic Experimental Design: Subjects are randomly assigned to either a control group or the experimental group *Post-Test Only Control Group Design: Participants are randomly assigned to either the control group or the experimental group and receive only a posttest *Quasi-Experimental Designs: A design lacking random assignment between groups receiving an intervention or where the researchers have little or no control over the allocation of the intervention that is to be studied. These types of studies can be useful in areas where it is not possible to conduct a fully randomized experiment such as with educational interventions, social program evaluations or public health interventions. There are limitations that come with this approach, in particular the ability to make causal inferences e.g., the results often can not be used to make an inference of cause and effect as there may be other variables this group of subjects are exposed to and could account for the results. However, there are various statistical techniques that can be used to account for other variables that may be present, which can improve the accuracy of the results *Nonequivalent Control Group Design: A design in which a group of persons who are similar in composition to the group receiving the intervention is used as a comparison group; for example, this group may receive similar pretest and posttest observations, but no treatment *Time Series Design: A design involving three equally spaced observations prior to the intervention and three afterwards; observations may exceed three, but the amount before the intervention must be equal to the amount afterwards *Multiple Time Series Design: Similar to a time series design, but includes a control group INDEPENDENT VARIABLE A design in which the variable is manipulated in some form by the researcher; the presumed cause INFORMED CONSENT Prospective research participants must be fully informed about procedures and risks involved in participating in a research study. Consent is given with that full and complete knowledge META-ANALYSIS An analysis combining the results of several studies that address a set of related research hypotheses. “The traditional meta-analysis is conducted after there is a sufficient number of research findings on a specific topic. There is no accepted standard for the number of studies needed before conducting a meta- analysis; reviewers, however, would be unlikely to accept a meta-analysis with less than five studies.” (Hanson & Broom, 2005, p. 358) METHODOLOGY A theory and analysis of how research does and should proceed MORTALITY Loss of subjects from the evaluation (participants may move or drop out for many reasons) PLACEBO FACTORS Generally mild and positive effects experienced by people as a result of their exposure to an innocuous intervention QUALITATIVE RESEARCH This “… refers to the meanings, concepts, definitions, characteristics, metaphors, symbols and descriptions of things …” (Berg, 1995, p. 3) *Note: The biases and interpretations of the interviewer can then become an integral part of the investigation.) QUALITATIVE DATA COLLECTION Three forms include: *Observation: An approach that includes gaining rapport, engaging in ongoing observation, and taking field notes; the researcher actively participates via participant observation. *Personal Interviewing and Focus Groups: The researcher will engage participants one- on-one in a personal interview allowing for the possibility of eliciting in-depth information. In Focus groups, a group of respondents are guided through discussion and questions by a moderator. A skilled moderator can elicit information offered by members that may be even more useful than that obtained through individual interviews as discussions are stimulated by the comments and support of others in the group. *Use of Documents: Captures information through responses to a research instrument such as a questionnaire or survey. QUANTITATIVE RESEARCH Research that relies on counts and measures of things (Berg, 1995, p. 3) QUANTITATIVE DATA Data based on numbers and mathematics; used to make statistical inferences RANDOM SAMPLE A study sample in which each individual is chosen entirely by chance, so that that each individual has the same probability of being chosen at any stage during the sampling process RANDOM ASSIGNMENT An experimental technique for assigning study participants to different treatments or no treatment at all RELIABILITY A measurement of the consistency and dependability of an instrument or questionnaire in measuring some concept or phenomenon with accuracy. If an instrument is reliable, then administering it to similar groups would yield similar results. SAMPLING A subset of a population. Two examples: *Probability Sampling: a form of sampling in which the sample is carefully chosen according to certain principles so that it is representative of a larger population *Non-Probability Sampling: A form of sampling in which the sample is not scientifically determined and thus has unknown accuracy or precision SATURATION A situation in which data analysis begins to reveal repetition and redundancy and when new data tend to confirm existing findings rather that expand upon them. STATISTICAL TERMS OR CALCULATIONS USED IN RESEARCH Some examples: *Mean: the sum of all the values in a group divided by the number of values in that group *Median: the midpoint in a set of scores; the point at which half the scores fall above and half the scores fall below (the middle-most score) *Mode: the value that occurs most frequently *Variability: reflects how scores differ from one another (i.e. spread and dispersion) *Range: computed by subtracting the lowest score in a distribution from the highest score in a distribution *Standard Deviation: represents the average amount of variability in a set of scores; the average distance from the mean; the larger the standard deviation the larger average distance each data point is from the mean of the distribution TRIANGULATION Using more than one method to arrive at the same conclusion; using multiple methods to gather data rather than relying on one particular method to provide the answer VALIDITY A situation in which an instrument closely corresponds to the concept it was designed to measure. Simply put, a valid test is a test that measures what it is supposed to. There are several ways that validity can be measured including: *Content Validity: A form of validity demonstrated if the entire range of the concept is represented in the sample of items selected for the scale. This is used when the researcher wants to know whether a sample of items reflects an entire universe of items in a certain topic; *Face Validity: A form of validity used when researchers analyze an instrument and agree that it appears to measure the concept *Criterion Validity: A form of validity that correlates the scores from the test with some other measure that is already valid and that assesses the same set of abilities *Concurrent Validity: A form of validity involving the administration of the new scale to the subjects, along with another scale that previous studies have shown to be a valid measure of the same concept *Construct Validity: A form of validity used when you want to know if a test measures some underlying psychological construct; sometimes involves the known-groups technique where the investigator administers the instrument to two very different kinds of groups expecting to find major differences in the way they respond THREATS TO INTERNAL VALIDITY Some examples: *Maturation: Because some problems improve as a result of the passage of time, researchers might erroneously conclude that an intervention was effective, when in actuality the subjects receiving the intervention matured or the passing of time served to make the problem less acute. *Testing: If a researcher is using a design where the same test is administered sequentially a number of times, those receiving the intervention may show improvement in their scores as a result of figuring out the “correct” response on the test. *Instrumentation: Just as those enrolled in a program can become bored by taking the same test on numerous occasions, the evaluator or other persons making observations might subtly or unconsciously modify the procedures. NOTE: If an instrument can be empirically demonstrated to have validity, then it can generally be assumed to have adequate reliability. However, a reliable instrument may not be valid. References Terms that appear in this glossary have been adapted from these sources: Berg, B. L. (1995). Qualitative research methods for the social sciences. Boston, MA. Allyn and Bacon. Brayton, J. (1997). What makes feminist research feminist? The structure of feminist research within the social sciences. Retrieved from the University of New Brunswick: http://www.unb.ca/PAR-L/win/ feminmethod.htm Padgett, D. K. (2008). Qualitative methods in social work research (2nd ed.). Thousand Oaks, CA: Sage. Royse, D., Thyer, B. A., & Padgett, D. K. (2010). Program evaluation: An introduction (5th ed.). Belmont, CA: Wadsworth CENAGE Learning. Salkind, N. (2008). Statistics for people who think they hate statistics (3rd ed.). Thousand Oaks, CA: Sage. Hanson, R. K., & Broom, I. (2005). The utility of cumulative meta- analysis: application to programs for reducing sexual violence. Sex Abuse: A Journal of Research and Treatment, 17, 357-373. doi:10.1177/107906320501700402 Sexual assault doesn’t cause binge drinking in girls: Study. (2011, July 29). HealthDay News. Retrieved from U.S. News & World Report: http:// health.usnews.com/health-news/family-health/childrens-health/ articles/2011/07/29/sexual-assault-doesnt-cause-binge-drinking-in-girls- study © National Sexual Violence Resource Center 2012. All rights reserved. This document was supported by Cooperative Agreement # 5VF1CE001751-03 from the Centers for Disease Control and Prevention (CDC). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the CDC.