Using Existing Measures

It is easy to get caught up in the idea of using measures that have been validated through research studies (e.g., the Illinois Rape Myth Acceptance Scale) as your tools of choice. While there are definite benefits to this (see below), there are also significant drawbacks. As with all things in evaluation, using existing measures should be done very deliberately and should not be considered the default option when planning your evaluation.

Benefits

• Existing measures have often undergone rigorous testing to ensure that they are actually measuring what they claim to be measuring (validity) and that they can do so repeatedly (reliability). As a result, using them strengthens your case for saying you are measuring the change you seek to measure with them.

• The hard work of drafting, re-drafting, revising, and re-revising questions is already done for you! Writing good questions is an art and a science and is far more difficult than most people imagine. Previously validated measures that have been developed by experts are likely not to fall prey to common pitfalls in survey development and also save you a lot of time in development.

• The use of existing measures can sometimes increase the buy-in from other partners, especially funders.

Drawbacks and Cautions

• Not all measures that are widely used or found in the research are actually valid and reliable.

• Even if they are valid and reliable, they may not be appropriate for the community or population with whom you are working. For example, many instruments are validated through research conducted with college populations. In contrast, many (if not most) prevention workers are working with younger populations. More than just being at a different educational level, the developmental level also influences how concepts and constructs are understood. The measures we use need to be appropriate to our given populations.

• In addition to being appropriate to our given population, our measures need to be specific to the change we are interested in creating (i.e., our outcomes) and related to the intervention(s) we are employing to make that change. Sometimes existing measures are chosen for the above benefits without sufficient consideration to their direct applicability to the work being measured; if they don’t speak to your outcomes and programming, then the benefits are irrelevant because the data won’t be useful to you.

• Measures can become outdated, especially measures that seek to assess particular social constructs like gender-role socialization. These constructs change over time, as do the indicators of them. For example, the Attitudes Toward Women Scale (Spence, Helmreich, & Stapp, 1973) includes the following item: “It is ridiculous for a woman to run a locomotive and for a man to darn socks.” The short version of this measure is from 1978. Nearly 40 years later, most Americans probably assume that it’s ridiculous for anyone to darn socks, but that’s not necessarily because there’s been a drastic shift in our cultural perceptions about gender.

• Starting with validated measures and editing the questions to fit your audience or issue is not necessarily a bad idea and might very well be better than starting from scratch. However, it is important to remember that changing anything about the instrument means it is no longer valid and reliable and is then ultimately an untested instrument

Additional Resources

Want more guidance on selecting, adapting, and evaluating prevention approaches? Check out this guide from the Division of Violence Prevention. This guide supports good decision making that balances delivering prevention approaches as intended with considering unique community contexts.

Measures Database (PDF, 3 pages) This database maintained by the Wisconsin Coalition Against Sexual Assault (WCASA) includes resources where you can find free measures, scales, or surveys specific to sexual assault prevention work. Some measures/scales are general examples and others are "standardized measures". Many examples are provided; there are pros and cons to each measure and WCASA does not endorse any specific options. Please contact NSVRC at prevention@nsvrc.org for assistance in identifying appropriate measures.

Evidence-Based Measures of Bystander Action to Prevent Sexual Abuse and Intimate Partner Abuse Resources for Practitioners (PDF, 31 pages) This document is a compendium of measures of bystander attitudes and behaviors developed by the Prevention Innovations Research Center. Some of the versions of the measures have been researched more thoroughly in terms of psychometric properties than others. Please see the citations provided for articles that describe the versions of our measures that have been published. See also, Evidence-Based Measures of Bystander Action to Prevent Sexual Abuse and Intimate Partner Violence: Resources for Practitioners
(Short Measures) (PDF, 22 pages) which provides administrators of prevention programs with shortened, practice-friendly versions of common outcome measures related to sexual abuse and intimate partner violence. These measures have been analyzed to develop a pool of scales that are concise, valid, and reliable.

Listen to NSVRC's Resource on the Go Podcast, How to Measure a Sense of Community (Audio, 17 minutes) where, NSVRC’s Evaluation Coordinator, Sally J. Laskey, talks with researchers Iris Cardenas, a PhD Candidate in the School of Social Work at Rutgers University, and Dr. Jordan Steiner about the Brief Sense of Community Scale and their study that examined the cultural relevance of the Brief Sense of Community Scale with non‐Hispanic, Black, and Hispanic college students.

References

Spence, J. T., Helmreich, R., & Stapp, J. (1973).  A short version of the Attitudes toward Women Scale (AWS). Bulletin of the        Psychonomic Society, 2, 219-220. doi:10.3758/BF03329252