This publication was supported by Cooperative Agreement #5VF1CE001751-02 from the Centers for Disease Control and Prevention. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention. National Sexual Violence Resource Center Prevention Assessment Year 1 Report: National Strengths and Needs Assessment 20 October 2010 Prepared By: Stephanie M.Townsend, PhD Program Evaluator & Consultant 8 Locke Drive, Pittsford, NY 14534 stephanie.townsend@earthlink.net (585)690-9315 Submitted To: National Sexual Violence Resource Center 123 N. Enola Drive Enola, PA 17025 Table of Contents Executive Summary 3 Methodology Evaluation Questions 5 Evaluation Design and Methodology 5 Procedures: Year 1 7 Measures: Year 1 8 Data Analysis 9 Findings Samples 10 What the Field Thinks About Prevention Endorsement of Principles 11 Definitions of Prevention 12 Beliefs About Prevention 16 Summary 23 What Programs Are Doing For Prevention Prevention Activities 24 Partnerships 27 Facilitators of Prevention Work 33 Barriers to Prevention Work 35 Evaluation 38 Summary 41 Next Steps 42 References 45 Appendices A. Survey Recruitment 46 B. Focus Group Recruitment 49 C. National Surveys 54 D. Statistical Details 84 E. Evaluator’s Background 97 Executive Summary In Fall 2009 the National Sexual Violence Resource Center embarked on a three-year process of assessing the primary prevention training and technical assistance needs of state/territory coalitions, RPE coordinators and local rape crisis programs. This report provides a summary of the work completed during the first year of the assessment and the assessment findings to date. The purpose of this project is to: • Assess and prioritize primary prevention training and technical assistance needs, including identification of facilitators and barriers of high quality primary prevention • Develop recommendations for future strategic directions to measure primary prevention capacity among individuals, organizations and systems • Assess the need for resources in Spanish • Document and analyze changes that occur over the three year period, particularly in regard to organizational capacity to do primary prevention. While the project is intended to identify training and technical assistance needs, it is equally important that strengths and accomplishments also be documented as they can provide important guidance for future work. Understanding what is working well is also critical for expanding the reach of promising innovations. As detailed in this report, the major activities of the Year 1 assessment were the completion of a national survey and focus groups. Major findings from these data sources include: • At the state/territory level, current endorsement of principles of effective prevention was remarkably consistency with the definitions of prevention being advanced by the Centers for Disease Control and Prevention. At the local level, consistency was noticeably less widespread. • Overall, coalitions, RPE coordinators and local programs hold positive beliefs about primary prevention and those beliefs show significant improvement compared with when primary prevention first started being emphasized. • The distinctions between primary, secondary and tertiary prevention are seen as useful for bringing greater focus on changing behaviors, norms, systems and culture and facilitating collaborative planning and priority setting. • However, the distinctions are also seen as problematic because of inconsistencies that create confusion and funding challenges that have arisen with the emphasis on primary prevention. • Coalitions and RPE coordinators are highly engaged in primary prevention. In some states/territories that engagement is marked by high degrees of collaboration. In others there is more differentiation of roles and responsibilities, and the degree of communication and the clarity of roles varies. • At the local level, primary prevention is currently focused on social skills training, gender issues, and bystander empowerment. However, it is important to note that general rape awareness education continues to be a major focus of most programs. • Facilitators of primary prevention work that were identified included: information about prevention practices, networking around prevention, and access to and skills for research collaborative relationships with a wide array of partners, and support for addressing root causes of sexual violence were noted. • The role of evidence and evidence based practices is complex. The paucity of evidence was repeatedly cited as problematic, especially in light of pressure to use evidence based practices. This may be introducing premature constraints on the strategies used by local programs. • Other barriers to primary prevention work included the number of prevention staff, funding levels for prevention, evaluation skills, culturally specific materials, and skills for working cross-culturally. • Finally, it is important to note that at the local level, RPE-funded programs reported stronger beliefs that primary prevention is easy and clear; reported more use of bystander empowerment, changing norms campaigns, community mobilization, and community coalitions; and reported that retention of prevention staff is less of a barrier than did non-RPE funded programs. To build on the information and insights gained from the national survey and focus groups, the Year 2 assessment proposed in this report includes two major activities: • Interviews with exemplar organizations • Revisions to the NSVRC training and technical assistance satisfaction surveys A timeline is proposed in the final section of this report that will ensure all Year 2 assessment activities will be completed and findings reported to the NSVRC by the end of September 2011. Methodology Evaluation Questions The three-year assessment is designed to answer questions in five key areas: • Organizational Capacity for Prevention: What are the core components of capacity? What is the capacity at this time? What do programs need to strengthen their capacity? What can the NSVRC do to support growth and sustainability? How does capacity change over the next three years? • Partnerships: What are the facilitators of and barriers to effective collaborations between RPE coordinators and coalitions? What other partnerships are needed for community-wide responses? What are the facilitators and barriers of those partnerships? How do partnerships change over the next three years? • Primary Prevention: How do programs define prevention? How have those definitions changed in recent years? What are the most common primary prevention strategies and/or activities being used? What challenges and successes are programs experiencing? How are programs working with diverse cultural and linguistic communities? What is their ability/likelihood of using multilingual resources? How do primary prevention strategies and activities change over the next three years? • Diffusion of Innovations: What are exemplars of innovative prevention at the local and state or territory levels? How did those innovations come about? How and to whom are innovative practices spreading? What are the facilitators of and barriers to diffusion? • Evaluation and Research: How are programs evaluating their primary prevention work? What skills and resources do they need to do more useful and/or rigorous evaluations? How much access does the field have to research related to sexual violence prevention? What skills do they need to critically analyze and use research? How can synthesis and translation of research be most useful to the field? How do evaluation and use of research change over the next three years? Evaluation Design and Methodology Evaluation is best when it is based on multiple sources of information and multiple methods of measurement. This triangulation process reduces the propensity toward measurement error and strengthens the validity of findings (Rossi, Freeman, & Lipsey, 1999). By using multiple methods and informants, we can be more confident in drawing conclusions about complex social systems (Singleton & Straits, 2009). To answer the evaluation questions, five methods will be used over the next three years. A summary of how each method fits in with the overall evaluation design is found in Table 1 on the following page. Survey Surveys are useful when the focus is on a set of predetermined questions and the answers will be coded using numeric or a very narrow set of codes (Singleton & Straits, 2009). Self-reported information such as organizational characteristics, activities engaged in, and attitudes are well-suited to a survey format. However, it must always be remembered that there may be some differences between reported and actual behaviors. Methodology Table 1. Evaluation Design Year1 Year2 Year3 NationalSurvey X X FocusGroups X Interviews X Satisfaction Surveys X X During Year 1 of the assessment, a national survey of strengths and needs vis-à-vis primary prevention was administered. This survey will be repeated in Year 3 and analyses conducted to identify changes that occur over time. Additionally, during Years 2 and 3 a revised satisfaction survey will be developed and used to assess user satisfaction with training and technical assistance they receive from the NSVRC. Focus Groups Focus groups involve people from similar backgrounds who participate in a facilitated discussion on a specific topic. The benefits of using focus groups are that they generate a rich understanding of the participants’ experiences and beliefs, help in exploring new areas of evaluation, provide context and depth of understanding, and solicit interpretations from participants themselves. The group context has the additional benefit of mimicking the social context in which organizational decisions are made (Patton, 2002). During Year 1, a series of three focus groups was held during the RPE Grantees Meeting and National Sexual Assault Conference. Separate groups were held for: • RPE coordinators • State/Territory coalition staff • Local program staff Interviews Similar to focus groups,interviews can provide a rich understanding of participants’ experiences and beliefs. However, because they are conducted on a one-on-one basis, it is possible to go in more depth and to explore experiences and issues that an individual might be reluctant to share in a group setting. Even more than in focus groups, interviews allow the evaluator to see the topic from the perspective of the person being interviewed (Patton, 2002). Because of their in-depth and interactive nature, interviews are also an effective way of checking the validity of conclusions that the evaluator may draw from other sources of data (Singleton & Straits, 2005). Interviews will be used in Year 2 for an in-depth exploration of exemplar innovations in primary prevention. Taking a case study approach, organizations at the local and state/territory levels that are especially innovative and/or that seem to have overcome many of the challenges faced in the field will be studied to better understand what has supported their innovations and how they solved any problems or challenges they encountered. Methodology Procedures: Year 1 National Strengths and Needs Survey The national survey was developed collaboratively between the NSVRC, Centers for Disease Control and Prevention, PreventConnect, and the evaluator. Invitations to participate in the survey were distributed by the NSVRC. All state/territory coalitions and RPE coordinators were invited to participate in an e-mail sent by the NSVRC on March 30th. In mid-April a reminder message was sent. (See Appendix A.) For rape crisis programs, 343 programs were randomly selected from the list of programs that is maintained by the NSVRC. The sample constituted 21% of all known rape crisis programs in the country. This was determined to be a sufficient sample for representation and a feasible number in light of the available evaluation resources. Rape crisis programs were also invited to participate via an e-mail sent by the NSVRC on March 30th. For those programs for which there was no email contact information, the invitation and a hard copy of the survey were sent by mail. In mid-April a reminder message was sent. (See Appendix A.) There were two differences in the procedures used for state/territory coalitions and RPE coordinators versus rape crisis programs. First, due to concerns about the initially low return rate from rape crisis programs and the possibility that surveying during Sexual Assault Awareness Month posed a burden to them, a final e-mail was sent to them at the beginning of May letting them know that the deadline for their participation was extended to May 14th. Second, rape crisis programs were offered a $25 stipend as a thank-you for their time. Stipends were sent only to those programs that voluntarily identified themselves on the survey. All identifying information was separated from the surveys so that the data were de-identified. Surveys were primarily conducted online via Survey Monkey. However, programs were also offered the opportunity to complete the survey on paper and return it by mail. These procedures were intended to achieve a nationally representative sample. As will be discussed in the findings, the samples did appear to fairly represent the field. Focus Groups Focus groups were held during the 2010 RPE Grantees Meeting and National Sexual Assault Conference held in Los Angeles August 30— September 3, 2010. Three separate groups were held one for each of three groups: RPE coordinators, state/territory coalitions, and rape crisis programs. Notes were taken during the groups, but the discussions were not audio recorded. Groups were scheduled for 90 minutes and all groups took the full time allotted. Organizations were invited to participate in the focus groups through multiple mechanisms. RPE coordinators and state/territory coalition staff were sent an invitation by e-mail approximately one month prior to the meeting and conference. The RPE coordinator e-mail was sent through their listserve and the coalition e-mail was sent by the NSVRC directly to coalitions. Invitees were asked to sign up by sending a reply e-mail to the NSVRC. Twenty slots were available and sign-up was on a first-come-first-served basis. However, three seats were reserved for territories to ensure their access to participating. (See Appendix B for invitations.) For rape crisis programs, an open invitation flyer was supposed to be included in the conference packets upon check-in at the conference. However, due to a logistical oversight this did not occur. Therefore, announcements were made during the workshops that were part of the prevention track and flyers were distributed on tables during one of the plenary sessions. Additionally, NSVRC staff and the evaluator issued invitations and encouragement to attend by word of mouth during the conference. (See Appendix B for the flyer.) Focus groups were held on the site of the RPE meeting and conference. The RPE coordinators group was held over lunch immediately prior to the opening of the meeting. The coalition meeting was held during the final workshop session on the first day of the conference. The rape crisis group was held during the final workshop session on the second day of the conference. Although the groups during the conference did potentially conflict with workshops that would be of interest to potential participants, efforts were made to minimize the potential conflicts. Focus groups were facilitated by the independent evaluator. One member of the NSVRC staff did attend to serve as notetaker. Her presence was not observed to noticeably hamper candid discussion. The one limitation of the focus group procedures that should be noted was the fact that participation was limited to those who traveled to the RPE Grantees Meeting and/or National Sexual Assault Conference. Therefore, programs that either could not afford to or elected not to attend the meeting and conference did not have the opportunity to participate. Despite this limitation, the meeting and conference were the most viable settings for conducting focus groups with national representation. The fact that the conference had approximately 900 registrants indicates that this venue was a reasonable and effective choice for obtaining national representation. Measures: Year 1 National Survey Two written surveys were developed: one for RPE coordinators and coalitions and a second survey for rape crisis programs. The five main areas assessed by each survey were identical: • How organizations define prevention, what they are doing for prevention work, and successes they have had • Challenges organizations face in doing primary prevention • Beliefs the staff hold about primary prevention • Whom organizations partner with on prevention • How organizations evaluate their prevention work The major differences between the two surveys were that questions were tailored to the state/ territory or local contexts. For example, the state/territory survey asked about state/ territory-level partnerships (e.g., state education department, state domestic violence coalition, state child welfare system, state medical association, etc.) whereas the local level survey asked about local partnerships (e.g., local schools and school districts, domestic violence agencies, child welfare agencies, hospitals, private medical providers, SANE/SAFE providers, etc.). This approach allowed for the relevant areas to be assessed while keeping the surveys parallel so that comparisons can be made about similarities and differences at the state/territory and local levels. Most questions were closed-ended, but some questions were asked in a open-ended manner. The open-ended questions elicited surprisingly long and in-depth responses and, consequently, yielded rich insights. The open-ended questions mostly focused on how organizations envision the prevention of sexual violence, examples of their success, and how they have found it useful and problematic to distinguish between primary, secondary and tertiary prevention. These open-ended questions were triangulated with closed-ended responses about what the organization is doing for prevention work, settings in which they are working, and what the oranization thinks are important characteristics of prevention programming. This triangulation provided a check on whether the ideas organizations endorsed in principle aligned with how they described their actual work. Copies of the surveys are found in Appendix C. Focus Groups Each focus group began with an overview of the project, an introduction of the evaluator, and introductions of group participants. Group members were invited and encouraged to speak candidly and assurance of confidentiality on the part of the evaluator and NSVRC notetaker were made. Group members were also asked to keep comments confidential, but were reminded that confidentiality could not be guaranteed. Focal questions were similar across groups, although some group-specific questions were included and the reference points (e.g., state/ territory versus local contexts) were specific to each group. All groups were asked to speak about their own experiences and to allow their constituents to speak for themselves. Each group attended to this very closely and, consequently, the insights gained from each group reflected their own perspectives. RPE coordinators and coalitions were each asked about: • Role differentiation between RPE coordinators and coalitions • Prevention successes and challenges • Training and technical assistance priorities • Relationship with the CDC (RPE coordinators only) • Changes in beliefs about primary prevention (coalitions only) • Challenges and needs for culturally-specific prevention (coalitions only) Rape crisis centers were asked about: • Changes in their beliefs about primary prevention • Prevention successes and challenges • Training and technical assistance priorities • Needs for culturally-specific prevention Data Analysis Closed-ended survey responses were analyzed using appropriate descriptive, parametric and non-parametric statistics. Analyses were run using SPSS version 18.0. For ease of understanding, throughout the body of this report results will be presented using non-technical language. Statistical details can be found in Appendix D. Open-ended survey responses and focus group notes were analyzed qualitatively using conventional content analysis (Hsieh & Shannon, 2005). This technique describes a phenomenon, in this case participants’ experiences with primary prevention. Open-ended responses and focus group notes were reviewed, codes were developed to describe and organize their content, and those codes were subsequently sorted into meaningful themes. Exemplars of the themes were then identified. The remainder of this report presents the findings from the national survey and focus groups. Findings are organized into five areas: • Description of the sample • What the field thinks about prevention • What programs are doing for prevention • Facilitators of prevention work • Barriers to prevention work Finally, the next steps for the Year 2 assessment are described. Samples The findings presented in this report are based on two samples: national survey participants and focus group participants. National survey respondents at the state/territory level included 21 RPE coordinators and 26 coalitions from 42 states and 1 territory. Six states had responses from both the coalition and RPE coordinator. As shown in Figure 1, among the RPE coordinators who responded, 90% were located in departments of health. Among the coalitions that responded, 58% were sexual violence only coalitions and 42% were dual sexual and domestic violence coalitions. At the local level, national survey respondents included 72 complete surveys from 33 states. As shown in Figure 2, slightly more than half of programs reported receiving RPE funds; slightly more than half were dual sexual and domestic violence agencies; and slightly more than half served some combination of urban, suburban and rural settings. Findings:Think About Prevention Although it would have been advantageous to have more states where both the RPE coordinator and coalition responded and more respondents at the local level, for a national survey this was a good response rate (46% state/territory level, 21% local level). Additionally, there was good geographic representation. What The Field Thinks About Prevention Endorsement of Principles The first view we have on what the field thinks about prevention is the endorsement of the principles of effective prevention. On the survey, respondents were asked to rate how important different components are for their own agency’s prevention programming. These included the principles of prevention that had been promoted over the past few years by the CDC, NSVRC and PreventConnect in their training and technical assistance. Additionally, the principles of community mobilization, addressing different forms of oppression, and being connected to feminism were added to reflect principles of high value to the NSVRC. A response of “5” indicated that the respondent “strongly agreed” that the principle was an important part of their prevention programming. As shown in Figure 3, at both the state/territory1 and local levels there was very high endorsement of all the principles. It is interesting to note that the patterns closely follow one another, although at the local level the differences between high and low points are larger. The differences between the state/territory– and local-level are due to the fact that: • At the state/territory level, all principles (except feminism) were endorsed as neutral or higher. No one expressed disagreement with their importance. • At the local level, for each principle at least one respondent “strongly disagreed”. At each level, analyses were run to see if there were significant difference between certain types of organizations. At the state/territory level the only significant differences in endorsements between RPE coordinators and coalitions were for: • Being theory driven • Being systematically evaluated For both of these principles, the endorsements by RPE coordinators were stronger. At the local level, there were some significant differences: • RPE funded programs more strongly endorsed the importance of theory than non-RPE funded programs • Urban programs more strongly endorsed the importance of multiple methods, theory, and community mobilization than did rural or mixed setting programs. Additionally, the overall average of all principles were more strongly endorsed by urban programs. At the RPE Grantees Meeting, a brief summary of the survey results was presented. A number of people expressed interest in the responses to feminism. Therefore, additional analyses were run to see if there were any ways to identify which agencies were more likely to endorse connections to feminism. No significant differences were found between: • RPE coordinators and coalitions • Sexual violence and dual coalitions • Departments of health and other RPE coordinators • Sexual violence, dual, and multiservice local programs • RPE-funded and non-RPE funded local programs • Rural, urban and multi-setting programs The only significant association found at both the state/territory and local levels was an association with the importance of addressing different forms of oppression. The association was such that the higher the reported importance of addressing oppression, the higher the endorsement of feminism. It is interesting to note that the NSVRC staff expected that the average endorsement of feminism would be much lower than it was. In contrast, at the RPE meeting responses were mixed with some individuals expressing surprise that the endorsement was as high as it was and other that it was as low as it was. Definitions of Prevention In addition to responding to pre-specified principles, respondents were asked to give their own explanations of their vision for prevention of sexual violence and successes they have had in prevention. These questions were asked in order to triangulate the endorsement ratings. While it might be easy to assume that high endorsement of the principles of prevention indicate that organizations are “on board” with primary prevention, this would be a false conclusion if based solely on the endorsement ratings. For example, it is possible for an agency to do only general rape awareness education through didactic presentations yet do those presentations in multiple settings, rely on positive relationships in the community, match the awareness presentations to psychosocial development, evaluate them for outcomes such as an increase in awareness, and have the presentations be delivered by well-trained staff, to name but a few of the principles. The open-ended questions are, perhaps, a more accurate assessment of organizations’ understanding of prevention. When describing how they would explain prevention to local rape crisis programs, all but one state/territory level respondent gave answers that were consistent with the definitions being advanced by the CDC and other leaders in the field. Most answers were notable for their depth and sophistication. There were many direct and indirect references to CDC and other training documents with the social-ecological model and the Spectrum of Prevention being the most frequently emphasized. Coalitions and RPE Coordinators Three themes that were frequently seen in the state/territory level responses were: • Prevention of Perpetration: “We talk about primary prevention as activities that prevent the assault from happening in the first place, and the main focus is primary perpetration prevention. We emphasize the difference between this strategy and risk reduction and/or victim services. It’s about culture change, bystander behavior, and engaging men and boys in education and activities that prevent perpetration.” “Prevention is stopping a condition, incident or situation from occurring. Prevent first time perpetration when it comes to sexual assault.” • Social Change and/or Norms Change: “When speaking casually about primary prevention, I focus on norms change and social/cultural change, and point to examples such as our work breaking down gender stereotypes.” “The intent of our prevention work is to change the social context that allows sexual violence to happen. We are also very focused on asset development and positive youth development to eradicate sexual violence.” “[We focus] on asking communities to take the responsibility for awareness, education, etc. to change social norms, focusing on the media, the responsibility of adults to create a healthy, safe environment for children vis-à-vis modeling and listening, as well as working with systems and institutions to change the language and actions around sexual violence.” “We talk about it as organizing to change social norms that support violence that uses sex. We refer to the spectrum of prevention and talk about prompting policy and organizational practice/procedure changes that address the toxic environment that normalizes sexual harm.” “We try to explain prevention as creating a culture where violence and oppression of any kind is unacceptable. Implicit and explicit social norms support positive, healthy, and productive relationships between all people.” • Health Promotion: “The active, assertive process of creating conditions that promote well-being. Prevention is strengths-based and engages people as resources in the creation of change.” “Enhancing protective factors for sexual violence prevention, such as family and community efficacy, positive youth development opportunities, economic and social equality, etc., etc. is emphasized, as well as reducing risk factors.” While most responses fell into one of these categories, some responses included multiple categories. The preponderance of definitions fit into the social change and/or norms change category. However, at the local level the responses were a bit different. As shown in Figure 4, when describing how they would explain prevention to people in their communities, 52% of rape crisis programs gave answers that were consistent with the definitions being advanced by the CDC and other leaders in the field, although these typically had less detail and depth than the state/territory level responses. Another 34% gave explanations that presented prevention as being solely about awareness and/or risk reduction; 8% gave mixed responses and 7% were unclassifiable due to their vagueness. The following are examples of typical responses. • Consistent: “We approach prevention by developing strategies that should take place prior to sexual assault/violence occurring for the purpose of preventing it. Basically, stopping it before it happens.” “Prevention of sexual violence is intervening prior to the development of a sex offender. Increasing norms that do not accept or condone violence and increasing the protective factors in the individual, relationships, families, communities and society that reduce/end the rates of sexual violence or the development of an offender." “I would explain primary prevention as stopping the problem before it starts. We would stop the problem by giving the community the skills they need to be a good bystander.” “Education to children and pre-teens regarding violence, healthy relationships, gender roles, etc.” “Primary prevention involves individual, relationship, community and societal components. In our agency, it is defined as a thought or culture shift in thought processes around sexual assault. Addressing societies’ contributions through gender and rape culture.” “It means we focus on changing social norms that perceive and promote women as object, possession, and victim and that perceive men as having entitlement just because they are male, and that link male violence and sex. We focus on identifying and accessing groups at high-risk of offending and, using male role models and peers work with them to challenge themselves and their male peers to practice a different way of being a man. We focus on using men as allies to prevent other men from offending rather than trying to help women avoid being victims.” • Awareness: “Prevention includes: increasing awareness in the community about the presence, the issues, and the resources; providing education/outreach programs to increase awareness.” “Prevention of sexual violence means educating the community and providing awareness, the content of which must include various at risk scenarios as well as knowledge on how to avoid or extract one’s self from them.” “Educate not only the general public, but also law enforcement, and our legal system. Education is key. Also to inform the public about resources and help.” “We teach them ‘no means no’, how to cover your drink and to stay with people when going to parties, etc. We also believe that raising awareness throughout the community will help people realize sexual assault happens in every community. We are available for presentations and provide up to date statistics, facts and information on what to do if a sexual assault does occur and how our agency can help. We also educate the community on what they can do if they know someone who has been sexually assaulted.” “Awareness. The more people know about sexual violence and what to look for and how to possibly identify warning signs, the better chance we have to prevent sexual violence.” “Accurate information and undoing the myths surrounding sexual vioence, especially to young women.” • Mixed: “To teach children and adults what is a healthy relationship. How not to become a predator. How to identify sexual violence and remove oneself from a violent situation.” “Prevention starts with education to not only help potential victims keep themselves out of unsafe situations, but also educating bystanders to step in and help when they see a situation escalating or potentially dangerous.” • Unclassifiable: “We do community organizing events to help spread the word.” “By working within our community, promoting and participating in intervention, education, and advocacy to prevent sexual violence.” In thinking about the endorsement of principles and their own open-ended definitions of prevention, it appears that at the state/territory level there is a very clear understanding of primary prevention and strong endorsement of the principles of effective prevention. At the local level, while endorsement of the principles was still very high, the grasp of what primary prevention actually looks like was more tenuous. In terms of the three-year assessment plan, it is likely that we have reached a “ceiling effect” at the state/territory level in regard to endorsement of the principles. While this can be assessed again to ensure that there has been no significant decrease in endorsement, it is doubtful that any detectable increase is possible. At the local level, there is more room for increases in endorsement ratings, but it should be noted current ratings are high enough that to reach a statistically significant change the effect will have to be strong. Qualitatively, it will be interesting to track any changes in definitions. At the state/territory level there was already near-perfect consistency with the CDC’s definitions. However, it may be useful to see whether there are changes in how CDC documents and language are used versus the use of an agency’s own way of articulating prevention. At the local level, there is clearly potential for positive change with almost half of programs currently not articulating definitions that are consistent with the primary/secondary/ tertiary framework. Beliefs About Prevention While the previous findings speak to fundamental understandings of prevention, there is a separate issue of what people in the field believe about prevention and specifically about primary prevention. The NSVRC was keenly aware of the struggles that agencies have engaged in as they make the shift from awareness presentations with an emphasis on the number of people in the audience to primary prevention, skill-building and norms change with an emphasis on behavioral outcomes. It is important to understand what the current beliefs are. The national survey was also an opportunity to ask respondents to think retrospectively to what they believed “when primary prevention first started being emphasized.” The scales used to assess beliefs about primary prevention are what are called semantic differential scales. These scales present a target, in this case “primary prevention.” For that target a list of paired adjectives follows and respondents are asked to rate their beliefs or feelings using the paired descriptors. The closer the respondent puts their mark to one of the words or phrases, the more that word or phrase describes their beliefs. A mark in the mid-point indicates that the paired words or phrases both equally describe their beliefs. On the survey, 10 pairs were presented: • Impossible vs. Possible • Difficult vs. Easy • Frustrating vs. Rewarding • Confusing vs. Clear • New to our organization vs. What we were already doing • Not consistent with our mission vs. Consistent with our mission • Competing with services to survivors vs. Complementing services to survivors • About changing individuals vs. About social change • Only responsibility of rape crisis centers vs. Responsibility of whole community • Only the job of prevention educators vs. Everyone’s job at a rape crisis center As shown in Figure 5,: • The pattern of beliefs at the state/ territory and local levels are very similar both now and retrospectively. • There have been substantial increases in the positive beliefs about primary prevention at both the state/territory and local levels. These changes were statistically significant for all beliefs. The only belief for which the average rating was considerably lower than all other beliefs was the idea that primary prevention is “easy”. This is to be expected and should not be interpreted as a negative finding. What is important is that the ease ratings have significantly improved over time. Again, analyses were conducted to see if there were differences between certain types of organizations in their current beliefs about primary prevention. This was done to help identify if targeted training or technical assistance is needed in this area. At the state/territory level the only significant difference in current beliefs was for the belief about whether prevention is the responsibility of prevention educators or the entire agency: • Coalitions expressed significantly stronger beliefs about prevention being the responsibility of the entire agency than did RPE coordinators. At the local level, RPE-funded programs expressed significantly stronger beliefs than did non-RPE funded programs about: • Prevention being easy • Prevention being clear It must be kept in mind that the findings presented in Figure 5 are average scores. Averages can sometimes hide more nuanced differences that may be important when thinking about whether to tailor training and technical assistance. To shed light on this, a cluster analysis was run. This technique identifies whether there are distinguishable patterns in the responses that can be used to categorize respondents. The cluster analysis revealed that there were three patterns of beliefs about primary prevention: • Strongly positive beliefs • Moderately positive beliefs • Skeptical beliefs Interestingly, the proportion of agencies in each of these categories varied. As shown in Table 2: • Slightly more than half of agencies at the state/territory level were in the category of strongly positive beliefs. • No local programs were in the strongly positive category. • The majority of local programs fell in the category of moderately positive beliefs. • Notably more local programs were in the skeptical category than were organizations operating at the state/ territory level. These findings indicate that in terms of bringing about positive change in organization’s beliefs about primary prevention, much more work is needed at the local level than at the state/ territory level. While the above findings provide an interesting picture of broad beliefs, it was also important to understand the more specific reactions in the field to the distinctions between primary, secondary and tertiary prevention. The NSVRC was aware that there are not only confusions surrounding this framework, but also fundamental concerns and critiques about the applicability of a public health framework to sexual violence. Therefore, the survey asked open-ended questions about how agencies have found it useful to distinguish between primary, secondary and tertiary prevention and how they have found it problematic. Additionally, the strengths and challenges of agencies’ primary prevention work was an extensive part of the focus groups. The description below integrates these two sources of data to describe what the field sees as useful and as problematic about the distinctions between primary, secondary and tertiary prevention and, more generally, about doing primary prevention work. With the exception of the fact that that local programs were more likely than coalitions or RPE coordinators to say the distinctions are not useful, the types of responses were similar across all three groups. Useful Aspects Four aspects of the distinction were most commonly cited as useful. • The most dominant theme when talking about the usefulness of the primary prevention framework was that it brings greater focus on changing behaviors, norms, systems and culture and clarifying the difference between prevention and risk reduction. Responses that exemplify this attitude include: “It is useful to explain the difference between services, risk reduction, and true primary prevention. It helps to shift the focus from victims to perpetrators and begin the discussion about culture change.” “This distinction is crucial in the development of effective primary prevention programming. Many individuals in our community consider risk reduction to be primary prevention. It’s been useful to clarify this to our community and service providers because it has lead to a decrease in victim blaming and an increase in the focus on prevention programming for youth.” • Agencies also talked about how the frame-work helps them to tailor efforts, plan programs and set priorities. For example: “It is useful in as far as we talk about hat might be the best fit for a particular audience and discuss how we move through this paradigm shift from resource/referral/awareness education opportunities to more defined primary prevention strategies.” “The prioritization of primary prevention has helped us be more focused and structured with regard to who we fund and what they are allowed to do.” “It has been useful to define our responsibility as primary prevention. There is often pressure to do secondary prevention, so having made a clear policy decision to limit our agency's work to primary prevention has helped raise the profile of primary prevention.” • At the local level, some programs also talked about how the framework allows for a more empowering or positive approach: “The primary prevention approach is a much more empowering and positive approach in working with teens and community members. We are able to focus on what role men can play as well as other bystander populations.” “We have found it hugely useful because it honors the important role that all of our staff play in prevention.” • Finally, although more about the process than the framework itself, at the state/ territory level the emphasis on primary prevention and the planning process was described as promoting collaboration in field. For example: “The fact that funding is non-competitive between the states and based on population creates an environment of collaboration among states and a sharing of resources.” [In response to this comment, another focus group participant said:] “The little states live off the ideas and resources of big states.” In the focus groups the planning process was described as: “based on collaboration”, “requiring explicit articulation of what we are doing”, “requiring us to come together to plan within a public health framework”, “bringing home the importance of sexual violence” and “getting everyone on the same page”. While keeping in mind these positive views, it is also important to acknowledge and give serious consideration to the substantial challenges that are faced at both the state/territory and local levels. There were four main challenges identified on the surveys and in the focus groups. While the specific examples or manifestations of these issues differed between the constituent groups, it was striking that all four issues were articulate by all of the groups. • The most fundamental issue identified revolved around inconsistencies that create confusion and lead to conflicting messages about what is fundable with RPE monies. Specifically, inconsistencies were noted in: • How primary, secondary and tertiary prevention are defined, with the major issue being that these terms are used differently in regard to rape prevention than they are in regard to other public health prevention issues • The types of activities that constitute each type of prevention, with the major issue being which specific activities are fundable with RPE monies • Expectations of RPE grantees, coalitions and local progras for their roles and what they are allowed and not allowed to do with RPE funds, specifically, CDC project officers were cited as giving inconsistent answers over time and of there being inconsistencies between project officers. Staff from local programs said they want concrete examples of each type of prevention, but they recognize that it is difficult to account adequately for variations. The example given was of health fairs. Health fairs were described as usually being awareness/outreach when, for example, agencies hand out free items that have the hotline number written on it. However, the question was raised about whether a health fair could be a type of primary prevention if the agency instead was handing out My Strength campaign materials (a campaign that is widely considered to be primary prevention) such as postcards and posters from the campaign. This was used to highlight the fact that it’s not possible to give a definitive list of activities that constitute primary prevention. At the state/territory level, questions were raised about how to handle contexts where, in light of community readiness models, a community is only at the level of building awareness. How are programs to do the necessary awareness building in order to move into primary prevention if awareness raising activities are deemed not to be a part of primary prevention? In some states this is not seen as a problem so long as there is a clearly articulated plan for what comes after awareness building and how the program will move the community to a higher stage of readiness for prevention. However, in other states programs have been told that this necessary, preliminary work is not fundable with RPE funds. This example highlights the challenges of differences between states. While each state is free to set its own requirements so long as they are consistent with the federal requirements, this creates confusion and difficulties in networking at the national level when programs are operating under different rules about what is and is not fundable as primary prevention. Among RPE coordinators, the flexibility of states to implement their own requirements and to use RPE funds in ways that may be very different from other states was accepted. However, among coalitions there was some disagreement with this. This disagreement came out of a concern that the CDC does not intervene enough to protect coalitions in states where there are conflicts between the coalition and the department of health. Responses to the idea that states have independence included: “They have made massive changes around what we are doing with prevention. Why can’t they make changes with the roles [of departments of health versus coalitions]...They have power.” “They mandate other things, such as 50% of RPE funds for primary prevention and a state planning process and they approve the state plans.” The final inconsistency that was noted was inconsistency between the public health framework and a social change movement. It should be noted that this was a point of debate among RPE coordinators, some of whom identify this work as a social change movement and others who explicitly rejected that perspective. • Not surprisingly, funding was consistently identified as a major challenge. However, it was not simply a matter of needing more funding (although that need was evident), but also of how funding can intersect with the primary prevention framework to create more uncertainty for the sustainability of coalitions and local programs. For RPE coordinators, funding concerns had to do largely with the small amount of funding rape prevention receives. This was especially pronounced for those coordinators in departments of health. Most RPE coordinating offices have very low FTEs. Additionally, to avoid indirect costs, some coordinators reported allocating funds using designations that allow more money to go to the programs. However, that sets them to up do more work for which they are not funded. Many RPE coordinators reported spending considerable unfinanced time on RPE work. However, the issue for RPE coordinators was not simply the base level of funding but also how it compares to other public health issues. Competition for scarce resources and recognition was especially a concern for RPE coordinators in departments of health. They described their departments as not understanding RPE or the CDC language around primary, secondary and tertiary prevention (which differs somewhat from how these terms are used in disease prevention). As a consequence, RPE was described as “working in a vacuum” within departments of health. There was widespread agreement among RPE coordinators that the focus is on domestic violence with one coordinator even being told that “sexual violence is not worthy of attention” and many coordinators agreeing that “the RPE funding level is so small that it lacks credibility” within the department. This situation decreases the leverage RPE coordinators might otherwise have, as reflected in the inadequate practical resources they are allocated. For example, RPE coordinators reported that, unlike other issue areas in their departments, RPE does not receive clerical support, travel funds, public relations support, FTE allocations, workload relief, or even such daily necessities as office keys and permission to photocopy materials. Additionally, they have to “fly under the political radar” For coalitions there was a parallel issue of unrecognized and unfunded prevention work with coalitions reporting that the CDC and departments of health “do not give us credit” for prevention work they do because they are not explicitly funded for that work. Interestingly, while some RPE coordinators perceive their sacrifice of indirect costs as not being recognized by coalitions, the coalitions in kind described themselves as absorbing cuts so that programs do not suffer but many of them complained that the departments of health do not take a similar cut. (There were exceptions to this with some coalitions acknowledging that both they and their coordinators absorb the cuts.) The most serious concern about funding at the state/territory level was the “chilling effect” reported after a coalition was defunded for prevention work. Some other states reported that they have been “repeatedly threatened” with being defunded, and in some cases that the precedent has been explicitly cited. At the local level, there were serious concerns in many states about the funding formulas being used to allocate prevention funds. Many of the old formulas rewarded high audience numbers, even if it was a single-session program. Now with the emphasis on multi-session skill-building curricula, most programs cannot continue to see the same high numbers. Many funding formulas have not been changed to reflect this shift. Consequently, programs are told to emphasize “quality, not quantity” and not to worry if their numbers go down, but they do not trust these reassurances and/or this message does not get to their executive directors. Another systematic concern with funding was the division between prevention and intervention funds. As a consequence, local programs have to submit multiple funding proposals and budgets in order to access different funding streams, even if all the funding sources are allocated by the same pass-through agency. In one state the coalition reported that their programs have to submit five separate proposals and budgets to access all of the funds that the coalition administers, whereas in the past they only had to submit one. Consequently, as one coalition representative said and numerous agreed with, “Now I feel more like a bureaucrat than an advocate.” • The emphasis on primary prevention, in combination with the funding structures, has also created a sense of competition within many local programs where prevention is now competing with survivor services and outreach. This sense of competition for scarce resources was described by both local programs and coalition staff. For example: “Primary prevention is key to ending sexual violence. However, within the context of rape crisis centers, this is difficult because usually 98% of their efforts are direct service — thus primary prevention seems less important when you are focusing on immediate safety of clients.” “It has been problematic because of the fact that we have to turn down community partners who call wanting us to do a health fair or information booth. Since we have to do primary prevention, we cannot simply to people about our agency and what we do here anymore. I feel that people are slipping through the cracks that may have already been abused when we can’t let the community know about our services, even though we are preventing others from finding themselves in that situation with primary prevention.” “The distinction inevitably implies additional work or a redirection of work, taking on an additional activity when the resources for victims are not adequate.” “Secondary and tertiary modes of prevention have always been valuable, and continue to be valuable and necessary as part of a comprehensive focus on prevention. Identifying them as secondary or tertiary can leave the impression that they are not a part of prevention work.” • Finally, while substantial improvements were noted by many people, lingering resentments persist among coalitions and local programs about how they think they are viewed by the CDC. Comments that reflect this include: “I’m not a stupid person but I’m made to feel like it due to the jargon they use.” “We are talked down to by the CDC and told ‘you’ve been doing it wrong.’” “[Confusion about what constitutes primary, secondary and tertiary prevention] causes resistance, anger and resentment among programs who are made to feel like they haven’t been doing it right.” In summary, the current thinking about prevention in the field is marked by: • High endorsement of the principles of effective prevention • At the state/territory level, almost complete consistency with definitions of prevention being advanced by the CDC. • At the local level, approximately half of programs defining prevention in a way that is consistent with the CDC. • Overall, positive beliefs about primary prevention with the beliefs held now being significantly more positive than when primary prevention first started being emphasized. • Differences in how strong the positive beliefs are, with 52% of state/territory level agencies but none of the local programs holding strongly positive beliefs. Ways in which the distinctions between primary, secondary and tertiary prevention are seen as useful included: • Bringing greater focus on changing behaviors, norms, systems and culture • Clarifying the difference between prevention and risk reduction • Tailoring efforts, planning programs and setting priorities • Allowing a more empowering or positive approach • Promoting collaboration in the field However, there were also notable ways in which the distinction was seen as problematic, including: • Inconsistencies that create confusion and lead to conflicting messages about what is fundable with RPE monies; including inconsistencies in: • How activities are designated as primary or non-primary prevention • How building awareness fits in to increasing community readiness for primary prevention • Differences between states in their requirements surrounding primary prevention • Conflicts between public health and social change perspectives • Funding challenges, including: • Too little base funding • Relative levels of funding for RPE versus other public health issues • Unrecognized and unfunded prevention work • Threats of defunding coalitions • Competition within local programs between prevention, survivor services and outreach • Lingering resentments for how coalitions and local programs are treated/perceived by funders What Programs Are Doing For Prevention The first set of findings focused on what programs think about prevention. The next major area to consider is what they are actually doing. On the surveys, respondents were asked about what their own agency is doing to prevent sexual violence. The lists of activities were different for the state/territory level and the local level surveys. The differences were intended to capture the unique things that need to happen at each level and the different roles of agencies. Coalitions and RPE Coordinators Coalitions and RPE coordinators were presented with a list of 17 activities. As shown in Table 3, almost all activities had substantial engagement reported. The activities reported by the most agencies as being engaged in were: • Training rape crisis programs on primary prevention (89%) • Providing information on promising practices (77%) • Offering networking opportunities for prevention educators (75%) • Providing technical assistance to prevention programs (75%) • Training allied organizations and professionals (67%) • Disseminating research on prevention to the public and allied professionals (67%) The only two activities that were frequently reported as not being of interest were: • Mandating specific prevention curricula or activities (48%) • Conducting research on rape prevention (26%). Otherwise, all activities were reported by the vast majority of organizations as being engaged in, being planned, or of interest. This widespread interest in all of the prevention activities presents a dilemma for the NSVRC and other leaders in the field as it does not help in identifying training and technical assistance priorities. Therefore, the RPE coordinators and coalitions were each asked during the focus groups (as well as during a presentation of preliminary findings at the Resource Sharing Project meeting in May 2010) for help in identifying priority areas. The responses varied between the two groups and so are presented here separately to help the NSVRC tailor training and technical assistance to specific constituencies. Coalition Priorities: • Evidence: what the evidence base is, how to move forward in the absence of a strong evidence base, how to communicate with funders (including the CDC) about “evidence informed” practices, how to strengthen the evidence base through evaluation and/or collaborating with researchers, and how to help programs use evidence to improve programs • Community Mobilization: how to do it, how to measure it, how to write it into proposals and contracts, how to train programs on community mobilization, and how to tailor expectations and strategies for community mobilization to specific contexts (especially rural areas) • Buy In: how to help shift the “haters of primary prevention” into the “champions of prevention,” how to market prevention to rape crisis centers, development of RPE coordinators’ own leadership skills for navigating the divide between programs that support and those that resist prevention, how to help programs let go of exclusive reliance on school based programs, and models for continuing to do awareness/ outreach work while at the same time freeing up resources for primary prevention • Networking: development of a network among coalitions, venues for networking that include more than simply executive directors, and more visibility of emerging practices Findings: Prevention Strategies and Activities Table 3. State/Territory Level Activities Not Doing & Not Doing Planning Engaged Not Interested But Interested to Do in Doing Info re: Promising Practices 0% 8% 15% 77% Recommending Specific 4% 19% 21% 55% Curricula/Activities Mandating Specific 48% 29% 8% 15% Curricula/Activities Statewide Prevention 2% 10% 21% 67% Initiatives Training RCCs on Primary 4% 2% 4% 89% Prevention Training Allied Orgs/Profs on 2% 10% 21% 67% Primary Prevention Providing Networking for 2% 10% 13% 75% Prevention Educators 1:1 Technical Assistance 2% 15% 9% 75% for prevention Programs Educating Legislators 7% 26% 22% 46% Building Local Capacity 0% 17% 35% 48% for Evaluation Evaluating Local Prevention 4% 30% 26% 40% initiatives Disseminating Research to 0% 19% 15% 67% RCCs Disseminating Research to 0% 29% 35% 35% Public and/or Allied Profs. Conducting Research on 26% 48% 9% 17% Rape Prevention Bringing Together Rape 0% 21% 19% 60% Prevention and Health Orgs. Bringing Together Rape 0% 21% 23% 56% Prevention and Other Allied Orgs. Working with culturally 0% 27% 27% 46% Specific Programs RPE Coordinator Priorities: • Definitions: clarification of ambiguities around what constitutes primary prevention • Core competencies for prevention educators: identification of what they are • Evaluation of primary prevention: how to advise programs how to modify what they are already doing for evaluation to make it stronger and models for how to carry out evaluations within existing resources • Best practices: identification of what the best practices are, how to implement what we do know about best practices, and how to adopt what is working with other public health issues (e.g., HIV/AIDS) to sexual violence prevention Role Differentiation The questions about activities were also intended to determine what, if any, role differentiation there is between RPE coordinators and coalitions. While the NSVRC staff had previously identified clear ideas about their different roles, this differentiation was not born out by the survey data. Each activity was tested for significant differences between what RPE coordinators and coalitions said they were currently doing. These analyses found a significant difference for only one variable: educating legislators, where 65% of coalitions and 20% of RPE coordinators said they were engaged in this activity. However, it is important to note that 70% of RPE coordinators said they were either planning on educating legislators or interested in doing so. While these data indicate a lack of role differentiation, it must be noted that the survey asked about these activities at a broad level. It may be that coalitions and RPE coordinators have different roles within these activities. Therefore, this was discussed both at the RSP meeting when preliminary findings were shared and during the RPE coordinator and coalition focus groups. In general, both groups agreed that there is differentiation of roles. For example, a coalition might host and facilitate a training for local programs while the RPE coordinator provides the funding. Both are involved in training local programs, but they are responsible for different tasks. Interestingly, in both groups there was a clear recognition that the differentiation of roles plays itself out differently in different states/ territories. The descriptions offered by both groups reflect a spectrum of collaboration: • Collaborative relationships where responsibilities are shared (“collaborative from beginning to end”, “we have a strong team and we all do everything”) • Differentiation with good communication and clear roles • Differentiation with little communication and unclear roles This latter situation was described by both groups as problematic, but was most passionately talked about by the coalitions who described these situations as involving “manipulation”, “dishonesty”, “threats”, “power and control”, and “my money versus stewardship”. As one person explained, “Building healthy relationships is at the core of what we promote, but this is not demonstrated by the department of health in how they treat us.” Furthermore, coalitions see the CDC as culpable in this because they are perceived as having a responsibility to uphold the guidelines and are seen as failing to do so. Local Programs The survey for local programs also asked about the activities they are engaged in, planning to do, not doing but interested in, and not doing and not interested. The responses are found in Table 4. At the local level there was more clarity about priority areas. The activities reported by the most agencies as being engaged in were: • Rape awareness education (97%) • Social skills training (91%) • Gender issues training (69%) • Bystander empowerment (64%) Differences based on setting and type of funding were tested. These analyses found that: • Urban and rural programs were significantly more likely to be interested in or planning prevention coalitions, whereas mixed setting programs were more likely to already have coalitions. • RPE-funded programs were more likely than non-RPE funded programs to already be doing: bystander empowerment, changing norms campaigns, community mobilization, and prevention coalitions. The significant differences between RPE-funded and non-RPE funded programs is particularly striking as it provides evidence that RPE funds do contribute to more primary prevention work. This finding may be useful in advocating for increases to the RPE funds. In terms of priorities for training and technical assistance at the local level, the best data we have on this comes from the activities on the survey that programs said they not doing but interested in plus those they were planning to do. Training and technical assistance in these areas may help programs move from interest and planning into action. At least half of programs indicated interest in or were planning: • Culturally specific prevention strategies (68%) • Anti-oppression training (65%) • Mobilizing men (58%) • Public or organizational advocacy (57%) • Systems and organizational change (56%) • Changing norms campaigns (54%) • Coalition building around prevention (51%) Partnerships In addition to what programs are doing, it was also of interest whom they are partnering with in their work. This is especially critical for primary prevention where community mobilization is a critical factor. Again, separate lists were used for state/territory and local level surveys to reflect the different contexts in which the agencies do their work. Findings: prevention Strategies and Activities Table 4. Local Level Activities Not Doing & Not Doing Planning Engaged Not interested But Interested to Do in Doing General Rape Awareness 1% 1% 0% 97% Education Bystander Empowerment 0% 21% 14% 64% General Social Skills Training 1% 1% 6% 91% Gender Issues Training 3% 16% 13% 69% Media Literacy Training 7% 39% 16% 38% Anti-oppression Training 9% 48% 17% 26% Mobilizing Men 6% 39% 18% 37% Training Professionals to Do 3% 23% 26% 48% Primary prevention Changing Norms Campaigns 4% 31% 21% 43% for Prevention Community Mobilization for 1% 29% 19% 50% Prevention Coalition Building for 0% 30% 21% 49% Prevention Public or Organizational 3% 39% 19% 40% Policy Advocacy Systems and Organizational 4% 34% 21% 40% Change Prevention Strategies for 3% 32% 35% 29% Specific Communities or Cultural Groups RPE Coordinators and Coalitions What was most striking about state/territory level responses was how few widespread partnerships were reported, as shown in Table 5. While all partners were being engaged with by at least some agencies, the only partners that were reported to be “collaborating partners” (i.e., they meet regularly, engage in collaborative planning about prevention, and do some type of jointly run prevention strategies) by a majority of respondents were: • Coalition/RPE partnerships (77%) • Partnerships with domestic violence coalitions (53%) However, the latter partnerships are largely due to dual coalitions as 75% of dual coalitions reported domestic violence partnerships compared with only 42% of sexual assault coalitions. When thinking about training and technical assistance, it may be useful to focus on “supporting partners” (i.e., they meet at least occasionally, communicate about their prevention work, share ideas, but do not have jointly run prevention strategies) and “emerging partners” (i.e., they are beginning to talk about prevention and to form a relationship.) Training and technical assistance may help to move these partnerships into active collaborations. Three stakeholders were identified as supporting or emerging partners by at least half of agencies: • Faith communities (59%) • Culturally specific communities (58%) • Men’s groups (54%) Local Programs Like at the state/territory level, as shown in Table 6, all partners were reported as being engaged with, but very few demonstrated widespread engagement by a large number of programs. The most frequently identified collaborating partners were: • Domestic violence agencies (56%) • K-12 schools (41%) Again, to help prioritize training and technical assistance, supporting and emerging partners may be an effective place to focus. At the local level, only one stakeholder: K-12 districts (55%) were identified by at least half of programs as supporting or emerging partners. Together, these findings indicate that sexual violence organizations are quite isolated in their prevention work. This is not unique to the United States. A recent evaluation of a global sexual violence initiative found this same challenge. Findings: Prevention Strategies and Activities Table 5. State/Territory Level Partnerships Collaborating Supporting Emerging State Education Department 35% 30% 13% Men's Groups 13% 13% 41% LGBT Groups 11% 23% 21% Disabilities Groups 22% 22% 17% Faith Communities 11% 28% 30% Culturally Specific Communities 19% 21% 36% Civic/Service Organizations 13% 13% 20% State Domestic Violence Coalition 53% 27% 9% State Child Welfare System 17% 11% 9% State mental Health Organizations 20% 11% 11% State Medical Association 2% 16% 7% State Legislature 4% 22% 20% RPE Coordinating Agency OR 77% 9% 2% Coalition Table 6. Local Level Partnerships Collaborating Supporting Emerging Individual Schools: K-12 41% 27% 17% School Districts: K-12 24% 31% 24% Colleges/Universities 30% 28% 10% Student Organizations: K-12 18% 15% 22% Student Organizations: College 22% 16% 13% Men's Groups 11% 14% 17% LGBT Groups 12% 6% 15% Disabilities Groups 13% 17% 13% Culturally Specific Communities 15% 15% 21% Civic/Service Organizations 18% 21% 25% Domestic Violence Agencies 56% 18% 11% Child Welfare Agencies 29% 34% 10% Mental Health Services 31% 31% 13% Hospitals 29% 34% 12% Community-Based or Private 18% 25% 16% Medical Providers SANE/SAFE Providers 36% 25% 9% Local Health Department 27% 25% 15% Teen Pregnancy/Family Planning 13% 31% 13% groups Faith Communities 18% 21% 24% State/Territory Sexual Assault 35% 28% 12% Coalition State/Territory RPE Coordinating 35% 24% 9% Agency In summary, at the state/territory level, the best way to describe what coalitions and RPE coordinators are doing for prevention work is that they are either engaged in or want to be engaged in almost everything. On the one hand, this reflects very well on their commitment to prevention and the comprehensive work underway. On the other hand, it makes it difficult to identify priority areas for training and technical assistance. However, in the focus groups some priority areas were identified: Coalition Priorities: • More effective use of and expansion of the evidence base • Strategies for community mobilization • Strategies for increasing buy-in by reluctant programs • Networking opportunities for coalitions RPE Coordinator Priorities: • Clarification of prevention definitions • Identification of core competencies for prevention educators • Strengthening of internal evaluations by programs • Identification of best practices While there was no apparent role differentiation between coalitions and RPE coordinators on the survey, in the focus groups they did identify different roles they may play. What was most striking, however, were the descriptions offered by both groups of the spectrum of collaboration that exists between them, ranging from collaborative relationships to differentiation with good communication and clear roles, to differentiation with little communication and unclear roles. From the perspective of coalitions, there is a clear need for active intervention in states where there are conflicts. At the local level, there was more clarity about priority areas with four activities being most frequently engaged in: • Rape awareness education • Social skills training • Gender issues training • Bystander empowerment Additionally, there was interest in developing more activities in the areas of: • Culturally specific prevention strategies • Anti-oppression training • Mobilizing men • Public or organizational advocacy • Systems and organizational change • Changing norms campaigns • Coalition building around prevention Facilitators of Prevention Work Recognizing the combination of common practices and unique innovations in approaches to prevention raises questions about what is facilitating or hindering this work. On both the state/territory level and local level surveys, respondents were asked about 25 items that could either support prevention work or be barriers to prevention work. The items were categorized in terms of: • Materials and skills • Relationships • Resources and organizational capacity To make it easier to interpret the findings in a meaningful way, the responses to these questions were subjected to a type of analysis called factor analysis. What this analysis does is search for patterns in the responses to see if there are groups of items for which respondents give similar answers. If there are, then those groupings are examined to see if they constitute meaningful categories. The factor analysis took all 25 items and found five categories of barriers/supports: • Research: access to research on prevention, skills for understanding and using research, access to crime or other databases, and skills for using online technology • Information: basic information on rape prevention and practical ideas for how to do primary prevention • Networking: information about what other programs are doing, opportunities to network with other prevention programs, representation of needs to the CDC, representation of prevention to allied professionals, and relationship with coalition/RPE coordinator • Organizational capacity: skills and resources for evaluating agency’s programs, strategies for retaining staff, number of prevention staff, agency-wide involvement in prevention, and organizational leadership around prevention, funding for prevention • Culturally specific prevention: prevention materials in Spanish, in other languages, that are oral or non -literacy based, and for working with specific cultural communities The averages for each category were recoded to indicate whether each program reported, overall, that the category was a barrier or a support. As shown in Figure 6, for all groups the most notable categories of support were: • Information about prevention • Prevention networks • Access to and skills for research Additional insight into facilitators of prevention was gained from the focus groups where each group was asked about what has helped to move their work forward to more emphasis on primary prevention. Each group spoke to their own experiences and roles, yielding unique sets of facilitators. Coalitions: • Explicit distinctions being made between risk reduction, awareness, changing community norms, etc. • The prevention planning process, especially insofar as it has brought in non-traditional partners who bring new resources and has allowed coalitions to put sexual violence on the radar of those partner organizations Figure 6. Reported Supports Coalitions RPE Coordinators Local Programs Research 84% 57% 78% Information 76% 96% 93% Networks 68% 67% 81% Org. Capacity 44% 23% 52% Culturally Specific 20% 23% 46% • Articulation of prevention as a part of coalitions’ own missions and creating an environment where prevention is membership-driven rather than imposed from outside • The more recent positive shift where the CDC and departments of health are recognizing coalitions as the leaders in sexual violence prevention and respecting their decades of work RPE Coordinators: • RPE as federally funded helps with hiring and brings stability to RPE at the state/territory level • Prevention planning processes sparked greater collaboration and planning within a public health framework • Continuous training at the local level, especially when local programs inspire and challenge each other to do more/better prevention work • Emphasis on root causes of sexual violence and social norms from a lifespan/intergenerational perspective that includes women and men, adults and youth • In some states, new relationships with departments of education and inclusion of sexual violence in their safety and injury programs Local Programs: • Organizational leadership on the part of executive directors and boards of directors that support prevention on equal footing with intervention • Being given the time to do background work before developing a specific program or prevention strategy and then sufficient time for development before implementation • Longstanding community relationships in combination with active collaboration and sharing of expertise • Extent to which community partners push for evidence based programs — the less they require evidence based programs, the easier it is to tailor programs to the local needs and context and to be more comprehensive; the more they want evidence based programs, the more likely agencies are to use “canned programs that may not fit the context...and that are usually narrow in what they focus on.” Although each group had its own set of facilitators, there were common themes in their responses. Clearly, collaborative relationships with a wide array of partners is critical at all levels. Whether internal to the field (e.g., relationships between the CDC, RPE coordinators, coalitions and local programs) or with allied organizations, collaborative partnerships bring resources to the work and help integrate sexual violence prevention into other settings. This is critical to community mobilization and systemic change. Support for addressing root causes of sexual violence was also a recurring theme. This type of support manifests in different ways for each group and sometimes comes in surprising ways. The reflections by local programs that less emphasis on evidence based programs allows for more comprehensive programs is important to heed. Evidence, evaluation and research are critically important. However, because the evidence base we have in sexual violence prevention is so limited, it is important to be attentive to unintended negative consequences of requiring evidence based practices (or of narrowly defining what constitutes evidence). As one prevention educator commented, “If all we have is Safe Dates and bystander empowerment, we don’t really have much to work with.” Because there are far more root causes of sexual violence than we have evidence based programs for, the reliance on evidence based programs has the potential to constrain prevention work in ways that may limit impact. Barriers to Prevention Work The final major area explored in the surveys and focus groups were barriers to prevention work. While the earlier findings were about how the emphasis on primary prevention may be problematic, the following findings focus more on practical barriers that are faced once the decision to engage in primary prevention is made. As shown in Figure 7, for all groups the most notable categories of barriers were: • The lack of culturally-specific skills and resources • Organizational capacity for prevention work The responses were examined for any significant differences based on type of agency. The only significant difference found at the state/territory level was that RPE coordinators reported research as a barrier significantly more often than did coalitions. At the local level there were no significant differences based on whether or not the programs received RPE funds and type of setting (i.e., urban, rural, mixed). Figure 7. Reported Barriers Coalitions RPE Coordinators Local Programs Research 16% 43% 22% Information 24% 5% 7% Networks 32% 33% 19% Org. Capacity 56% 77% 49% Culturally Specific 80% 77% 54% Because organizational capacity and culturally specific skills and resources were identified as the greatest barriers by all respondent groups, these were examined more closely. The specific aspects of organizational capacity that are barriers varied depending on the group. As shown in Figure 8: • The two strongest barriers for all groups were number of prevention staff and funding levels. • Coalitions and RPE coordinators also reported substantial barriers related to evaluation; the fact that this was not frequently reported as a barrier for local programs is likely related to their lack of knowledge about the importance and usefulness of evaluation and may not reflect actual greater capacity for evaluation. • RPE coordinators reported more organizational barriers than either coalitions or local programs. The differences between RPE coordinators and coalitions were significant for agency-wide orientation and organizational leadership. It is notable that significantly fewer RPE-funded programs reported the retention of staff as a barrier than did non-RPE funded programs. This is further evidence of the positive impact of RPE funding. No other significant differences were found based on RPE funding or setting. A similar breakout was done for barriers to doing culturally specific prevention work. As shown in Figure 9: • All aspects of culturally specific materials and skills were rated as barriers by a similar proportion of respondents. Figure 8. Barriers to Organizational Capacity Coalitions RPE Coordinators Local Programs Evaluate 60% 64% 33% Retain 32% 64% 30% # Staff 68% 86% 61% Agency Orientation 28% 55% 26% Org. Leadership 16% 64% 20% Funding 72% 82% 71% Figure 9. Barriers to Culturally-Specific Prevention Coalitions RPE Coordinators Local Programs Spanish 52% 50% 41% Other Languages 60% 55% 48% Oral 56% 59% 43% Culturally Specific 60% 73% 49% Skills 56% 64% 43% • However, local programs rated these aspects as barriers less frequently than did coalitions or RPE coordinators. This does raise some questions about how extensively local programs would use materials if they were available. However, the percentage of programs reporting these aspects as barriers is still quite high. Although culturally-specific programming was on the agenda for the focus groups, due to time constraints there was little discussion about this issue. However, a more extensive discussion did take place during the presentation of preliminary results at the RSP meeting. There, the emphasis was on skill development. Without the necessary skills for working with culturallyspecific communities, access to materials was seen as having little impact. Furthermore, the types of skills that were identified focused on working cross-culturally. Given the relatively few prevention staff in most programs and the wide range of cultural groups in the communities they serve, this must be the focus of training and technical assistance, as opposed to training on how to work with specific communities. This point is further reinforced by the fact that programs reported as many as six languages or cultural groups for which they need materials. The average number was two. Languages and cultural groups named included: • Spanish (77%) • Native American (11%) • Russian (11%) • Chinese (9%) • Arabic (6%) • Blind and visually impaired (6%) • Deaf and hard of hearing (6%) • Filipino (6%) • Laotian (6%) • People with intellectual disabilities(6%) • Somali (6%) • Vietnamese (6%) • African American (3%) • Bantu (3%) • French (3%) • Incarcerated people (3%) • Indian (3%) • Korean (3%) • LGBTQ (3%) • Men (3%) • Rural communities (3%) • Sudanese (3%) There clearly is a pressing need for prevention resources in Spanish. However, the diversity of needs also speaks to the complexity of providing culturally-specific materials. Evaluation While mentioned earlier in the context of organizational capacity, the final area explored was the need for training and technical assistance on program evaluation. Anticipating that this would be an area of need, specific questions on the surveys asked about what agencies currently do to evaluate their prevention work and their confidence for doing basic evaluation tasks. Half of coalitions and RPE coordinators and slightly more than half (57%) of local programs said they evaluated their prevention programs in the last year. As shown in Figure 10, the most common method used was surveys. Notably fewer local programs reported using interviews or focus groups. This is a potential area for targeted training and technical assistance, especially in light of the fact that these qualitative methods do not require statistical skills to analyze the data and they are among the more cost-efficient methods programs can use. Although not a part of this evaluation, the evaluator’s experience with providing training and technical assistance to local programs indicates that while programs are very comfortable with using surveys, their skills at writing surveys that effectively measure the outcomes they are interested in is minimal. Additionally, their ability to analyze survey data in a way that contributes to program improvement and planning is in need of strengthening. However, both of these skills can be greatly improved through training and technical assistance. Figure 10. Evaluation Methods Used Coalitions RPE Coordinators Local Programs Surveys 100% 100% 96% Interviews 57% 60% 44% Focus Groups 71% 80% 35% Observations 67% 80% 63% Archives 43% 40% 31% Figure 11. Outcomes Measured Coalitions RPE Coordinators Local Programs Satisfaction 88% 100% 91% Knowledge 88% 50% 93% Attitudes 75% 75% 86% Intents 63% 50% 64% Behaviors 38% 25% 39% As for what programs are measuring with their evaluations, as shown in Figure 10, very few agencies reported measuring behavioral intents or actual behaviors. This is a critical area for improvement for coalitions, RPE coordinators and local programs alike. Finally, confidence at completing some basic evaluation tasks was assessed. As shown in Figure 12, confidence with evaluation was quite low with the percentage of people stating they were “very confident” ranging from 0% - 60%, depending on the task. The fact that local programs reported greater confidence than coalitions and RPE coordinators for many of the evaluation tasks may be due to coalitions and RPE coordinators having more insight into what they actually entail. For example, if a respondent thinks that analyzing quantitative data simply means calculating percentages and averages, they may rate themselves higher than a respondent who is aware of the possibility of statistical analyses such as testing for statistically significant differences between groups and over time, and correlations, and other statistical techniques. Regardless of what respondents may or may not have considered when they completed the survey, the data about evaluation indicate a need for substantial training and technical assistance on all aspects of evaluation. Figure 12. "Very Confident" with Evaluation Tasks Coalitions RPE Coordinators Local Programs Logic Models 32% 43% 27% Defining Goals 60% 41% 58% Design 24% 5% 35% Measures 32% 9% 35% Quantitative 20% 18% 35% Qualitative 12% 0% 30% Using Findings 40% 23% 50% In summary, facilitators of primary prevention work that were identified on the survey included: • Information about prevention practices • Networking around prevention • Access to and skills for research Additionally, focus group discussions identified some specific facilitators. Across groups, the common themes were: • Collaborative relationships with a wide array of partners (including those facilitated by the prevention planning process) • Supports for addressing root causes of sexual violence It is important to note that some factors that may facilitate effective work can also have unintended, negative consequences. This is particularly true for the emphasis on evidence based practices where, in light of the paucity of research on sexual violence prevention, requirements for evidence based programs can be constraining and work against comprehensive programming. The strongest barriers to prevention work are in the areas of organizational capacity and culturally-specific materials and skills. Of particular concern are: • Number of prevention staff • Funding levels for prevention • Evaluation skills • Widespread and diverse needs for culturally specific materials, especially in Spanish • Skills for working cross-culturally As described in the Methodology, the evaluation activities described in this report were the first part of a three-year assessment. With the information and insights gained from the national survey and focus groups, the assessment can move forward into the next phase. Year 2 is designed to include two major activities: • Interviews with exemplar organizations • Revisions to the NSVRC training and technical assistance satisfaction surveys Each of these activities is described below, followed by a proposed timeline for the project. Interviews with Exemplars The primary assessment activity for Year 2 will be interviews with exemplars of excellence and innovation in primary prevention. These exemplars will serve as case studies in prevention. The interviews are intended to provide rich, detailed insights into: • The nature of the innovations (e.g., types of activities or strategies used, how the innovations fit the needs and context, potential for adapting the innovation to other contexts, etc.) • How the innovations developed (e.g., catalysts, development process, key contributors, changes over time, use of theory or evidence, etc.) • Components of organizational capacity that were critical to the successful development and/or implementation of the innovations • The role of partnerships in the development and/or implementation of the innovations • How, if at all, the innovators share their programs, models, resources, etc. with other organizations • Findings from any evaluations that have been done of the innovation The interviews will be conducted by the evaluator with multiple representatives of the organizations, as applicable. For example: prevention educators, program administrators, executive directors, etc. Interviews will be conducted via telephone and online technologies. If opportunities present themselves for site visits, these will be done. While site visits are ideal for gaining the richest insights, because of the cost associated with them they are not included as an integral part of the assessment. In addition to the interviews, exemplar organizations will be asked to share pertinent documents that may shed further light on the innovations (e.g., curricula, evaluation reports, campaign materials, etc.) In light of the budget for this project, the target is to complete case studies of six exemplar organizations.1 Cases will be selected in collaboration with the NSVRC using purposive sampling. The goal is to have cases that include innovations that: • Are culturally specific • Are implemented within school settings in a way that allows for intensive, developmentally appropriate skill building • Are implemented outside of school settings • Reflect both programmatic and organizational innovation Satisfaction Surveys Currently, the NSVRC has a process for assessing satisfaction with the technical assistance provided. However, the surveys can be revised to provide more useful feedback and contribute to ongoing improvement and planning. In collaboration with NSVRC staff, the current satisfaction surveys and processes will be reviewed and revisions to the measures and procedures developed. This will provide the NSVRC with ongoing evaluation data. Additionally, the results of the revised surveys will be used in the Year 2 and Year 3 assessment to monitor the technical assistance being provided. A similar process will be used for improving the measures used to assess outcomes of NSVRC trainings. A proposed timeline for these activities is found in Table 7. This timeline will be revised in collaboration with NSVRC staff. All Year 2 assessment activities will be completed by September 30, 2011, with the report of findings presented to the NSVRC at that time. Contributions to Research The NSVRC may want to consider how the Year 2 and Year 3 assessments may contribute to the published research. The case studies from Year 2 and the Year 1—Year 3 comparisons are likely to yield information that can be useful not only to the NSVRC but also to a wider audience that includes researchers studying community responses to sexual violence, prevention practices, and community/social change; public health administrators, and policy makers. Therefore, in addition to sharing findings with the field through its own mechanisms, the NSVRC may want to plan for publications in academic journals. This will require that the measures and procedures be approved by an Institutional Review Board to ensure that standards for the protection of human subjects are met. If the NSVRC is interested, the November planning meeting can include a discussion of options for this review. Table 7. Year 2 Assessment Timeline (Proposed) Assessment Activities November 2010 Planning meeting with NSVRC staff • discussion of Year 1 results • case selection, initial discussion of interview protocol • discussion of and planning for satisfaction surveys December 2010 Development of interview protocol Final case selection Development of revised satisfaction survey Development of training survey January 2011 Recruitment for case studies begins Implementation of revised satisfaction survey begins Implementation of training survey begins February—May 2011 Interviews for case studies conducted (completed by May 30) Compilation of interview and archival data from exemplar cases Implementation of revised satisfaction survey continues Implementation of training survey continues June—August 2011 Analysis of all case study data Implementation of revised satisfaction survey continues Implementation of training survey continues (August) Satisfaction survey data and training survey data for January— July 2011 provided to evaluator September 2011 Analysis of all case study data completed Analysis of revised satisfaction survey and training survey completed Year 2 findings reported to NSVRC References Minkler, M., & Wallerstein, N. (2003). Community-based participatory research for health. San Francisco: Jossey-Bass. Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd edition). Thousand Oaks: Sage. Reinharz, S. (1992). Feminist methods in social research. New York: Oxford University Press. Rossi, P. H., Freeman, H. E., & Lipsey, M. W. (1999). Evaluation: A systematic approach (6th edition). Thousand Oaks: Sage. Singleton, R. A., & Straits, B. C. (2009). Approaches to social research. New York: Oxford University Press. Appendix A: Survey Recruitment Initial E-Mail Recruitment for Coalitions and RPE Grantees The National Sexual Violence Resource Center (NSVRC) is conducting a national evaluation of strengths and needs of organizations doing rape prevention and education work. Your organization is invited to contribute to this evaluation by completing an online survey. The survey is confidential and will take approximately 45 minutes to complete. All surveys must be completed by April 25th to be included in the results. You can access the survey online at: https://www.surveymonkey.com/s/NSVRCstate If you prefer to complete the survey on paper, you may call the NSVRC at 1-877-739-3895 (toll free) to request a paper version be mailed to you. If you wish to take the survey over the phone, call the NSVRC at 1-877-739-3895. If you have any questions about the survey you may contact: at the NSVRC: Jennifer Grove, 877-739-3895 x. 121, jgrove @nsvrc.org the evaluator: Stephanie Townsend, PhD, 585-690-9315, Stephanie.townsend@earthlink.net We know that your time is valuable. Your input is vital. Sharing your experiences and ideas will help to shape strategic planning, training and technical assistance at the national level and will support the work of all rape crisis and prevention programs. Thank you for considering this request and thank you for the important work you do. Initial E-Mail Recruitment for Rape Crisis Centers The National Sexual Violence Resource Center (NSVRC) is conducting a national evaluation of strengths and needs of organizations doing rape prevention and education work. Your organization is invited to contribute to this evaluation by completing an online survey. The survey is confidential and will take approximately 45 minutes to complete. All surveys must be completed by April 25th to be included in the results. If you choose to participate, your organization will receive a $25 stipend as a token of our thanks. You can access the survey online at: https://www.surveymonkey.com/s/NSVRCrcc If you prefer to complete the survey on paper, you may call the NSVRC at 1-877-739-3895 (toll free) to request a paper version be mailed to you. If you wish to take the survey over the phone, call the NSVRC at 1-877-739-3895. If you have any questions about the survey you may contact: at the NSVRC: Jennifer Grove, 877-739-3895 x. 121, jgrove @nsvrc.org the evaluator: Stephanie Townsend, PhD, 585-690-9315, stephanie.townsend@earthlink.net We know that your time is valuable. Your input is vital. Sharing your experiences and ideas will help to shape strategic planning, training and technical assistance at the national level and will support the work of all rape crisis and prevention programs. Thank you for considering this request and thank you for the important work you do. Follow-up E-Mail Recruitment for Coalitions and RPE Grantees Hello, A couple of weeks ago your agency received an invitation to participate in a national survey being done by the National Sexual Violence Resource Center (NSVRC). If you have already completed the survey, thank you! If you have not yet completed the survey, we encourage you to do so. You can find the details in a copy of the original invitation that is pasted below. We know that Sexual Assault Awareness Month (SAAM) is a very busy time. We encourage you to think of this survey as a way of participating in SAAM at the national level. Sharing your agency’s experiences and ideas is a way of speaking out for what your state/territory needs to prevent sexual violence and to promote safe and equitable relationships. Thank you, NSVRC Staff Follow-up E-Mail Recruitment for Rape Crisis Centers Hello, A couple of weeks ago your agency received an invitation to participate in a national survey being done by the National Sexual Violence Resource Center (NSVRC). If you have already completed the survey, thank you! If you have not yet completed the survey, we encourage you to do so. You can find the details in a copy of the original invitation that is pasted below. We know that Sexual Assault Awareness Month (SAAM) is a very busy time for many agencies. We encourage you to think of this survey as a way of participating in SAAM at the national level. Sharing your agency’s experiences and ideas is a way of speaking out for what your community needs to prevent sexual violence and to promote safe and equitable relationships. Thank you, NSVRC Staff Appendix B: Focus Group Recruitment Prevention of Sexual Violence: Strengths, Challenges and Future Directions — Focus Group for RPE Coordinators RPE Coordinators have an opportunity to shape the prevention of sexual violence at the national level. Your input is needed to determine the direction national leaders will take for training, technical assistance and other support for the prevention work being done in your state/territory and communities. As part of the NSVRC’s national strengths and needs assessment, we will be holding a focus group for RPE coordinators. The discussion will explore: • The roles of RPE coordinators and state/territory coalitions in providing training and technical assistance on prevention • Successes and challenges in the prevention of sexual violence • Training and technical assistance needs and priorities This group is only for RPE coordinators. It will be held: Monday, August 30, 2010 Noon—1:30pm (lunch will be provided) At the RPE meeting site If you would like to participate, contact Jennifer Grove at the NSVRC at jgrove@nsvrc.org or (717) 909-0710. Space is limited to the first 20 people who respond. A waitlist will be kept in case there are cancellations. Because space is limited, we ask that only one person from each state/territory attend. If you are not able to attend but would like to offer input, you may speak with the focus group facilitator who is leading the assessment project, Stephanie Townsend, during the conference or contact her at stephanie.townsend@earthlink.net or (585)690-9315. There will be separate focus groups held during the conference for state/territory coalition staff and local rape prevention programs. Coalitions will receive invitations in advance of the conference. Local programs will receive an invitation in their conference registration packet. Feel free to encourage your coalition and program partners to attend their respective groups. Prevention of Sexual Violence: Strengths, Challenges and Future Directions — Focus Group for Coalition Staff Coalition staff have an opportunity to shape the prevention of sexual violence at the national level. Your input is needed to determine the direction national leaders will take for training, technical assistance and other support for the prevention work being done in your state/territory and communities. As part of the NSVRC’s national strengths and needs assessment, we will be holding a focus group for state/territory coalition staff. The discussion will explore: • What has facilitated changes in prevention • The roles of coalitions and RPE coordinators in providing training and technical assistance on prevention • Successes and challenges in the prevention of sexual violence • Training and technical assistance needs and priorities, including for the development of culturally-specific prevention efforts This group is only for coalition staff. It will be held: Wednesday, September 1, 2010 During the National Sexual Assault Conference Time & Location will be announced If you would like to participate, contact Jennifer Grove at the NSVRC at jgrove@nsvrc.org or (717) 909-0710. Space is limited to the first 20 people who respond. A waitlist will be kept in case there are cancellations. Because space is limited, we ask that only one person from each coalition attend. If you are not able to attend but would like to offer input, you may speak with the focus group facilitator who is leading the assessment project, Stephanie Townsend, during the conference or contact her at stephanie.townsend@earthlink.net or (585)690-9315. There will be separate focus groups held during the RPE meeting for RPE coordinators and during the conference for local rape prevention programs. Local programs will receive an invitation in their conference registration packet. Please encourage your programs to attend their respective focus group. Prevention of Sexual Violence: Strengths, Challenges and Future Directions — Focus Group for Rape Crisis & Prevention Staff Rape crisis center staff and prevention educators have an opportunity to shape the prevention of sexual violence at the national level. Your input is needed to determine the direction national leaders will take for training, technical assistance and other support for the prevention work being done in your state/territory and communities. As part of the NSVRC’s national strengths and needs assessment, we will be holding a focus group for staff of rape crisis centers and rape prevention programs. The discussion will explore: • What has facilitated changes in prevention • Success and challenges in the prevention of sexual violence • Training and technical assistance needs and priorities, including for the development of culturally-specific prevention efforts This group is only for rape crisis centers and rape prevention programs. It will be held: Thursday, September 1, 2010 Time Location If you would like to participate, sign up at the NSVRC table. Space is limited to the first 20 people who respond. A waitlist will be kept in case there are cancellations. Because space is limited, we ask that only one person from each program attend. If you are not able to attend but would like to offer input, you may speak with the focus group facilitator who is leading the assessment project, Stephanie Townsend, during the conference or contact her at stephanie.townsend@earthlink.net or (585)690-9315. There will be separate focus groups held during the RPE meeting for RPE coordinators and during the conference for coalition staff. Announcement to be Read During Prevention Track Workshops: In addition to learning about prevention in workshops such as this one, there are also opportunities for you to give your input into the future direction of prevention in our movement. On Wednesday and Thursday there will be two discussions held, one for state/territory coalition staff and the other for rape crisis center staff and prevention educators. Space is limited. If you are interested in either of these discussions, please stop by the NSVRC table to learn more and to sign up. Appendix C: National Surveys National Strengths and Needs Survey (State) The National Sexual Violence Resource Center (NSVRC) is conducting a national evaluation of strengths and needs of organizations doing rape prevention and education work. We especially want to learn about what the trends are for primary prevention, what helps organizations in their prevention efforts, and what the unmet needs are. This information will help the NSVRC better coordinate efforts at the national level and better support local programs, state coalitions and RPE coordinating agencies. Your organization is invited to contribute to this evaluation by completing an this survey. All state /territory coalitions and RPE coordinators are being invited to participate in this survey. The survey is confidential. The only person who will see your individual answers is the independent evaluator who has been contracted by the NSVRC to lead the evaluation. Neither the NSVRC staff, any state coalition staff, or any funders will know which organizations completed the survey. The evaluator will prepare a summary of the results for the NSVRC staff who will share results with the field. The survey will take approximately 45 minutes to complete. All surveys must be completed by April 25th to be included in the results. If you prefer to take the survey online, call the NSVRC at 877-739-3895 and request that the link be sent to you via email. We ask that the survey be completed by the person in your organization who is primarily responsible for coordinating rape prevention and education programming, but that person may receive input from others in the organization if needed. The survey will cover four areas: 1. Prevention Strategies: how you define prevention, what you think prevention should include, and what your organization is doing 2. Experiences with Primary Prevention: your agency’s views on primary prevention, barriers and supports for prevention work 3. Partnerships: who you are partnering with, successful partnerships, and challenges 4. Evaluation of Prevention Efforts: how your organization evaluates its prevention work If you have any questions about this survey you may contact: at the NSVRC: Jennifer Grove, 877-739-3895 x. 121, jgrove@nsvrc.org the evaluator: Stephanie Townsend, PhD, 585-690-9315, stephanie.townsend@earthlink.net We know that your time is valuable. Your input on these issues is vital. Sharing your experiences and ideas will help to shape strategic planning, training and technical assistance at the national level and will support the work of all rape crisis and prevention programs. Thank you for considering this request and thank you for the important work you do. You may return your survey by mailing it to: Stephanie Townsend, PhD 8 Locke Drive Pittsford, NY 14534 ORGANIZATIONAL BACKGROUND In order to understand how the needs of different organizations are similar and different from one another, we would like some basic information about your organization. Reminder: The only person who will see your individual answers is the independent evaluator who has been contracted by the NSVRC to lead the evaluation. If you need to consult with other staff to answer these questions, you may do so. 1. What state or territory is your agency in? _____ 2. Is your agency a: _____ State or territory coalition _____ RPE coordinating agency IF you are a state or territory coalition: Is your coalition a: _____ Sexual assault coalition _____ Dual sexual assault and domestic violence coalition IF you are an RPE coordinator: Are you located in a: _____ Department of Health _____ Attorney General’s Office _____ Governor’s Office or Governor’s Task Force _____ Other: ______________________________ 3 How many staff in your agency work primarily on sexual violence prevention? _____ Full-time employees _____ Part-time employees _____ Full-time interns _____ Part-time interns _____ Volunteers 4. In your state or territory are RPE funds awarded: _____ To all rape crisis programs _____ To only some rape crisis programs through a competitive process 5. In your state or territory are RPE funds awarded to non-rape crisis programs that do rape prevention work? _____ Yes _____ No PREVENTION STRATEGIES This first major part of the survey asks about how your agency defines primary prevention of sexual violence, what your agency is doing around prevention, successes you have had, and challenges you are facing. How Your Agency Defines Prevention 6. Please briefly describe how your agency envisions the prevention of sexual violence. For example, when you are talking with rape crisis centers about prevention, how do you explain it to them? 7. When you think about what your agency is doing to prevent sexual violence, what are you most proud of or what is an example of your success? 8. In what ways, if any, has your agency found it useful to distinguish between primary, secondary and tertiary prevention? 9. In what ways, if any, has your agency found it problematic to distinguish between primary, secondary and tertiary prevention? What Your Agency Does for Prevention 10. For the next questions, think about what your agency is doing to prevent sexual violence. For each activity please indicate whether your agency is currently engaged in the activity. Currently My agency is... Not doing Not doing Planning to Engaged in this & not this but do this doing this Interested interested Providing information on promising prevention practices Recommending specific prevention curricula or other activities Mandating specific prevention curricula or other activities Conducting statewide prevention initiatives Training local RCC/rape prevention programs on primary prevention Training allied organizations and professionals on primary prevention Providing networking opportunities for prevention educators Providing one-on-one technical assistance for prevention programs Educating legislators re: need for prevention funds, regulations, and/or root causes of sexual violence Building local programs’ capacity to evaluate their prevention initiatives Evaluating local programs’ prevention initiatives Disseminating research on rape prevention to rape prevention programs Disseminating research on rape prevention to the public and/or allied professionals Conducting research on rape prevention Bringing together rape prevention organizations and allied health organizations for coordinated efforts Bringing together rape prevention organizations and other allied organizations for coordinated efforts Working with culturally specific programs Other: ______________________________________ Of the prevention activities listed above, which one(s) do your staff spend most of their time doing? (List up to 3) 11. For the next questions, please rate how important you think each of the following components are for prevention programs in your state or territory. Then rate how confident you are in your agency’s ability to promote each component in the field. Our agency thinks this is an important I am confident that our agency can part of prevention programming... promote this in the field. Strongly Disagree Neutral Agree Strongly Strongly Disagree Neutral Agree Strongly Disagree Agree Disagree Agree Use multiple strategies Be done in multiple settings Use varied teaching methods Expose people multiple times to prevention messages Be driven by theory Rely on positive relationships in the community Be timed to match psychosocial development Be socially and culturally relevant By systematically evaluated for outcomes Be delivered by well trained staff Mobilize the entire community to engage in primary prevention Address different forms of oppression Be connected to feminism Information, Resources and Assistance Needs 12. Many factors may be influence your interest in and/or ability to do prevention work. For the following factors, please indicate how much they are a barrier or a support to your prevention work. Remember to rate what the current reality is, not what it would be if you had these things. This will help the NSVRC assess what the current needs are. For example, if not having a particular resource is a barrier to your work you would mark it as a “major barrier” or “minor barrier”. In contrast, if you have a particular resource and it is an important part of what makes your work possible, you would mark it as a “major support” or “minor support”. Major Minor Neutral Minor Major Barrier Barrier Support Support Materials or Skills: Basic information on rape prevention Practical ideas for how to do primary prevention Prevention materials in Spanish Prevention materials in other languages Specify: __________________________ Prevention materials that are oral or non-literacy based Prevention materials for working with specific cultural communities Which communities: _______________ Skills for working with specific cultural groups Access to research on prevention Skills to understand and use research Access to crime or other databases Skills for using online technology Relationships: Information about what other prevention programs are doing Opportunities to network with other prevention programs Leadership to bridge sexual and domestic violence movements Access to prevention experts Representation of our needs to the CDC Representation of prevention issues to allied professionals Relationship with our state/territory sexual assault coalition OR RPE coordinating agency (as applicable) Resources and Organizational Capacity Skills and resources for evaluating our programs Strategies for retaining prevention staff Number of prevention staff Agency-wide involvement in prevention Prevention staff not having other job responsibilities (e.g., direct services, fundraising, etc.) Organizational leadership around prevention Funding for prevention EXPERIENCES WITH PRIMARY PREVENTION Over the past few years there has been a push in the field toward using federal Rape Prevention Education funds for primary prevention. It will help the NSVRC to know what this process has been like for your agency so we can apply lessons learned in the future. 13. The following are some words and phrases that might be used to describe what your agency thinks about primary prevention. For each pair put one X to show how much you think the word/phrase describes your agency’s experience. (Reminder: You may get input from other staff if needed.) When primary prevention first started being emphasized, our agency thought primary prevention was… The closer you put your X to a word, the more that word describes your experience. All Left Word Equally Both All Right Word Impossible _____ _____ _____ _____ _____ _____ _____ Possible Difficult _____ _____ _____ _____ _____ _____ _____ Easy Frustrating _____ _____ _____ _____ _____ _____ _____ Rewarding Confusing _____ _____ _____ _____ _____ _____ _____ Clear New to our What we were organization _____ _____ _____ _____ _____ _____ _____ already doing Not consistent Consistent with our with our mission _____ _____ _____ _____ _____ _____ _____ mission Competing Complementing with services services to to survivors _____ _____ _____ _____ _____ _____ _____ survivors About changing About social individuals _____ _____ _____ _____ _____ _____ _____ change Only responsibility Responsibility of rape of whole crisis centers _____ _____ _____ _____ _____ _____ _____ community Only the job of Everyone’s job prevention at a rape crisis educators _____ _____ _____ _____ _____ _____ _____ center 14. Now our agency thinks that primary prevention is... All Left Word Equally Both All Right Word Impossible _____ _____ _____ _____ _____ _____ _____ Possible Difficult _____ _____ _____ _____ _____ _____ _____ Easy Frustrating _____ _____ _____ _____ _____ _____ _____ Rewarding Confusing _____ _____ _____ _____ _____ _____ _____ Clear New to our What we were organization _____ _____ _____ _____ _____ _____ _____ already doing Not consistent Consistent with our with our mission _____ _____ _____ _____ _____ _____ _____ mission Competing Complementing with services services to to survivors _____ _____ _____ _____ _____ _____ _____ survivors About changing About social individuals _____ _____ _____ _____ _____ _____ _____ change Only responsibility Responsibility of rape of whole crisis centers _____ _____ _____ _____ _____ _____ _____ community Only the job of Everyone’s job prevention at a rape crisis educators _____ _____ _____ _____ _____ _____ _____ center PREVENTION PARTNERSHIPS Because the prevention of sexual violence is a community-wide and multi-system effort, we would like to hear about the partnerships and networks that your organization works with. Who Your Partners Are 15. Please use the following definitions to identify the types of partnerships you have: • Collaborating Partners: You meet regularly, engage in collaborative planning about prevention, and do some type of jointly run prevention strategies • Supporting Partners: You meet at least occasionally, communicate about your prevention work, share ideas, but do not have jointly run prevention strategies • Emerging Partners: You are beginning to talk about prevention and to form a relationship • Other Partnership: You have a supportive or collaborative partnership but it is not focused on prevention • No Partnership: You have no formal connections You may choose more than one answer. Collaborating Supporting Emerging Other No Partners partners partners partnership Partnership State Education Department Men’s Groups LGBT Groups Disabilities Groups Faith Communities Culturally Specific Communities Civic/Service Organizations State Domestic Violence Coalition State Child Welfare System State Mental Health Organizations State Medical Association State Legislature RPE Coordinating Agency OR State/ Territory Sexual Assault Coalition (as applicable) Other: _______________________ 16. What are the strengths of the collaboration between your state/territory’s sexual assault coalition and RPE coordinating agency? 17. What challenges do you face in the collaboration between your state/territory’s sexual assault coalition and RPE coordinating agency? 18. What do you think most needs to change in the collaboration between your state/ territory’s sexual assault coalition and RPE coordinating agency? EVALUATION OF PREVENTION EFFORTS Evaluating prevention efforts may be challenging. In order to support evaluation efforts, the last section of this survey asks about your agency’s approach to evaluating your prevention work. 19. In the past year has your agency carried out any state– or territory-wide prevention campaigns or initiatives? _____ Yes (continue with question #20) _____ No (skip to question #23) 20. How do you know your prevention initiatives are successful at achieving the outcomes you want? How Your Agency Evaluates Its Prevention Efforts 21. During the past year, which of the following approaches has your agency used to evaluate its prevention work? ______ We have not evaluated our prevention work Who Lead Evaluation (select all that apply): ______ Our own staff led an evaluation ______ Someone outside our organization led an evaluation on a voluntary basis ______ We paid someone outside our organization to lead an evaluation ______ Our state coalition evaluated our work ______ Our Dept. of Health evaluated our work ______ Other (specify: _________________) Methods Used (select all that apply): ______ Survey (including pre-post surveys) ______ Interviews ______ Focus Groups ______ Observations ______ Archival Data (e.g., sexual harassment complaints, police records, etc.) ______ Other (specify: _________________) 22. Which of the following types of outcomes do you measure when you evaluate your prevention programs? ______ Participants’ satisfaction with the program ______ Knowledge about sexual assault (e.g., definitions, facts, etc.) ______ Attitudes about rape (e.g., rape myth acceptance, etc.) ______ Intent or likelihood of behaving in certain ways (e.g., intervening as bystanders, committing acts of violence, etc.) ______ Actual behaviors (e.g., actual interventions as bystanders, perpetration, etc.) 23. There are many approaches to evaluating program outcomes. The list below names some tasks that are often completed during an evaluation. Please rate how confident you are in your agency’s ability to train rape prevention programs or otherwise assist them in developing these skills. Very A Little Moderately Very Unsure Confident Confident Confident Develop logic models Define program goals and objectives Design an evaluation (e.g., figure out how to collect data) Develop or select a way to measure outcomes (e.g., surveys, interviews, focus groups, etc.) Analyze numerical/quantitative data Analyze open-ended/qualitative data Use evaluation findings to improve our work 23. Is there anything else the NSVRC needs to know about what is happening in your state or territory around prevention or issues you think need to be addressed at the national level? THANK YOU Thank you for taking the time to complete this survey. If you have any questions about this survey you may contact: at the NSVRC: Jennifer Grove, 877-739-3895 x. 121, jgrove@nsvrc.org the evaluator: Stephanie Townsend, PhD, 585-690-9315, stephanie.townsend@earthlink.net National Strengths and Needs Survey (RCC) The National Sexual Violence Resource Center (NSVRC) is conducting a national evaluation of strengths and needs of organizations doing rape prevention and education work. We especially want to learn about what the trends are for primary prevention, what helps organizations in their prevention efforts, and what the unmet needs are. This information will help the NSVRC better coordinate efforts at the national level and better support local programs and state coalitions. Your organization is invited to contribute to this evaluation by completing this survey. Your organization was randomly selected to receive this invitation. The survey is confidential. The only person who will see your individual answers is the independent evaluator who has been contracted by the NSVRC to lead the evaluation. Neither the NSVRC staff, any state coalition staff, or any funders will know which organizations completed the survey. The evaluator will prepare a summary of the results for the NSVRC staff who will share results with the field. The survey will take approximately 45 minutes to complete. All surveys must be completed by April 25th to be included in the results. If you choose to participate, your organization will receive a $25 stipend as a token of our thanks. Stipends will be mailed at the end of April. If you prefer to take the survey online, call the NSVRC at 877-739-3895 and request that the link be sent to you via email. We ask that the survey be completed by the person in your organization who is primarily responsible for coordinating rape prevention and education programming, but that person may receive input from others in the organization if needed. The survey will cover four areas: 1. Prevention Strategies: how you define prevention, what you think prevention should include, and what your organization is doing 2. Experiences with Primary Prevention: your agency’s views on primary prevention, barriers and supports for prevention work 3. Partnerships: who you are partnering with, successful partnerships, and challenges 4. Evaluation of Prevention Efforts: how your organization evaluates its prevention work If you have any questions about this survey you may contact: at the NSVRC: Jennifer Grove, 877-739-3895 x. 121, jgrove@nsvrc.org the evaluator: Stephanie Townsend, PhD, 585-690-9315, stephanie.townsend@earthlink.net We know that your time is valuable. Your input on these issues is vital. Sharing your experiences and ideas will help to shape strategic planning, training and technical assistance at the national level and will support the work of all rape crisis and prevention programs. Thank you for considering this request and thank you for the important work you do. You may return your survey by mailing it to: Stephanie Townsend, PhD 8 Locke Drive Pittsford, NY 14534 ORGANIZATIONAL BACKGROUND In order to understand how the needs of different organizations are similar and different from one another, we would like some basic information about your organization. Reminder: The only person who will see your individual answers is the independent evaluator who has been contracted by the NSVRC to lead the evaluation. If you need to consult with other people at your agency to answer these questions, you may do so. 1. What state or territory is your agency in? __________________ 2. What kind of community(ies) does your agency serve? _____ Only urban _____ Only suburban _____ Only rural _____ Combination 3. Is your agency a: _____ Stand-alone rape crisis agency _____ Dual rape crisis and domestic violence agency _____ Multi-service agency _____ Other: ________________________________ 4. How many staff in your agency work primarily on sexual violence prevention? _____ Full-time employees _____ Part-time employees _____ Full-time interns _____ Part-time interns _____ Volunteers 5. Does your agency receive federal Rape Prevention Education funds? _____ Yes _____ No _____ I Don’t Know 6. What other sources of funding do you use for your prevention activities? _____ State funds _____ Foundation funds _____ Private donors _____ Other: __________________________ 7. Approximately what percentage of your prevention budget is dedicated to primary prevention? ______ PREVENTION STRATEGIES This first major part of the survey asks about how your agency defines primary prevention of sexual violence, what is happening in your community around prevention, successes you have had, and challenges you are facing. How Your Agency Defines Prevention 8. Please briefly describe how your agency envisions the prevention of sexual violence. For example, if you were asked by someone in your community what it means to prevent sexual violence, how would you explain it to them? 9. When you think about what your agency is doing to prevent sexual violence, what are you most proud of or what is an example of your success? 10. In what ways, if any, has your agency found it useful to distinguish between primary, secondary and tertiary prevention? 11. In what ways, if any, has your agency found it problematic to distinguish between primary, secondary and tertiary prevention? 12. For the next questions, think about what your agency is doing to prevent sexual violence. For each item, please indicate whether your agency is currently using that strategy. Currently my agency is... Not doing Not doing Planning to Engaged in this & not This but do this doing this interested interested General Rape Awareness Education (rape myths, laws, risk reduction, how to help , etc.) Bystander Empowerment (how to intervene when witness rape culture or unsafe situations) General Social Skills Training (communication, assertiveness, healthy relationships, etc.) Gender Issues Training (gender stereotypes, gender roles, etc.) Media Literacy Training Anti-Oppression Training Mobilizing Men Training Professionals to Do Primary Prevention as part of their own work Changing Norms Campaign for Prevention (e.g., Men of Strength, Choose Respect, Green Dot, etc.) Community Mobilization for Prevention Coalition Building for Prevention Public or Organizational Policy Advocacy Systems and Organizational Change Prevention Strategies for Specific Communities or Cultural Groups Identify communities/groups:___________________ Other: ______________________________ 13. Of the strategies listed above, which one(s) do your prevention staff spend most of their time doing? (List up to 3.) 14. When training other groups/professionals to do primary prevention work as part of their own roles and responsibilities, do you specifically tailor your trainings for any of the following audiences? Yes No Parents Health Care Providers Social Service Providers Educators Clergy/Religious Leaders Other: _________________ 15. When you work in schools, what are you doing in addition to or instead of classroom presentations? 16. Which of the following settings outside of schools are you doing prevention work in? Yes No Youth/Community Recreation Centers Faith Communities Neighborhoods Workplaces Entertainment Settings (e.g., bars, clubs, etc.) Youth Sports Organizations Girl/Boy Scouts Social Service Agencies Other: _____________________ How Your Agency Approaches Prevention 17. For the next questions, please rate how important each of the following components are for your agency’s prevention programming. Then rate how confident you are in your agency’s ability to use each component in its own prevention programming. For our agency this is an important part of I am confident that our agency can do this our prevention programming... if it chooses to. Strongly Disagree Neutral Agree Strongly Strongly Disagree Neutral Agree Strongly Disagree Agree Disagree Agree Use multiple strategies Be done in multiple settings Use varied teaching methods Expose people multiple times to prevention messages Be driven by theory Rely on positive relationships in the community Be timed to match psychosocial development Be socially and culturally relevant By systematically evaluated for outcomes Be delivered by well trained staff Mobilize the entire community to engage in primary prevention Address different forms of oppression Be connected to feminism Information, Resources and Assistance Needs 18. Many factors may be influencing your interest in and/or ability to do prevention work. For the following factors, please indicate how much they are a barrier or a support to your prevention work. Remember to rate what the current reality is, not what it would be if you had these things. This will help the NSVRC assess what the current needs are. For example, if not having a particular resource is a barrier to your work you would mark it as a “major barrier” or “minor barrier”. In contrast, if you have a particular resource and it is an important part of what makes your work possible, you would mark it as a “major support” or “minor support”. Major Minor Neutral Minor Major Barrier Barrier Support Support Materials or Skills: Basic information on rape prevention Practical ideas for how to do primary prevention Prevention materials in Spanish Prevention materials in other languages Specify: ________________________ Prevention materials that are oral or non-literacy based Prevention materials for working with specific cultural communities Which communities: _______________ Skills for working with specific cultural groups Access to research on prevention Skills to understand and use research Access to crime or other databases Skills for using online technology Relationships: Information about what other prevention programs are doing Opportunities to network with other prevention programs Leadership to bridge sexual and domestic violence movements Access to prevention experts Representation of our needs to the CDC Representation of prevention issues to allied professionals Relationship with our state/territory sexual assault coalition Resources and Organizational Capacity: Skills and resources for evaluating our programs Strategies for retaining prevention staff Number of prevention staff Agency-wide involvement in prevention Prevention staff not having other job responsibilities (e.g., direct services, fundraising, etc.) Organizational leadership around prevention Funding for prevention EXPERIENCES WITH PRIMARY PREVENTION Over the past few years there has been a push in the field toward using federal Rape Prevention Education funds for primary prevention. It will help the NSVRC to know what this process has been like for your agency so we can apply lessons learned in the future. 19. The following are some words and phrases that might be used to describe what your agency thinks about primary prevention. For each pair put one X to show how much you think the word/phrase describes your agency’s experience. When primary prevention first started being emphasized, our agency thought primary prevention was… (Reminder: You may consult with others at your agency if you need to.) The closer you put your X to a word, the more that word describes your experience. All Left Word Equally Both All Right Word Impossible _____ _____ _____ _____ _____ _____ _____ Possible Difficult _____ _____ _____ _____ _____ _____ _____ Easy Frustrating _____ _____ _____ _____ _____ _____ _____ Rewarding Confusing _____ _____ _____ _____ _____ _____ _____ Clear New to our What we were organization _____ _____ _____ _____ _____ _____ _____ already doing Not consistent Consistent with our with our mission _____ _____ _____ _____ _____ _____ _____ mission Competing Complementing with services services to to survivors _____ _____ _____ _____ _____ _____ _____ survivors About changing About social individuals _____ _____ _____ _____ _____ _____ _____ change Only responsibility Responsibility of rape of whole crisis centers _____ _____ _____ _____ _____ _____ _____ community Only the job of Everyone’s job prevention at a rape crisis educators _____ _____ _____ _____ _____ _____ _____ center 20. Now our agency thinks that primary prevention is... All Left Word Equally Both All Right Word Impossible _____ _____ _____ _____ _____ _____ _____ Possible Difficult _____ _____ _____ _____ _____ _____ _____ Easy Frustrating _____ _____ _____ _____ _____ _____ _____ Rewarding Confusing _____ _____ _____ _____ _____ _____ _____ Clear New to our What we were organization _____ _____ _____ _____ _____ _____ _____ already doing Not consistent Consistent with our with our mission _____ _____ _____ _____ _____ _____ _____ mission Competing Complementing with services services to to survivors _____ _____ _____ _____ _____ _____ _____ survivors About changing About social individuals _____ _____ _____ _____ _____ _____ _____ change Only responsibility Responsibility of rape of whole crisis centers _____ _____ _____ _____ _____ _____ _____ community Only the job of Everyone’s job prevention at a rape crisis educators _____ _____ _____ _____ _____ _____ _____ center PREVENTION PARTNERSHIPS Because the prevention of sexual violence is a community-wide and multi-system effort, we would like to hear about the partnerships and networks that your organization works with and what makes those partnerships successful and/or challenging. Who Your Partners Are 21. Please use the following definitions to identify the types of partnerships you have: • Collaborating Partners: You meet regularly, engage in collaborative planning about prevention, and do some type of jointly run prevention strategies • Supporting Partners: You meet at least occasionally, communicate about your prevention work, share ideas, but do not have jointly run prevention strategies • Emerging Partners: You are beginning to talk about prevention and to form a relationship • Other Partnership: You have a supportive or collaborative partnership but it is not focused on prevention • No Partnership: You have no formal connections You may choose more than one answer. Collaborating Supporting Emerging Other No partners Partners Partners Partnership Partnership Individual Schools: K-12 School Districts: K-12 Colleges/Universities Student Organizations: K-12 Student Organizations: College Men’s Groups LGBT Groups Disabilities Groups Culturally Specific Communities Civic/Service Organizations Domestic Violence Agencies Child Welfare Agencies Mental Health Services Hospitals Community-Based or Private Medical Providers SANE/SAFE Providers Local Health Department Teen Pregnancy/Family Planning Groups Faith Communities State/Territory Sexual Assault Coalition State/Territory RPE Coordinating Agency Other: _______________________________ 22. What are the strengths of your community prevention partnerships? 23. What challenges do you face in your community prevention partnerships? EVALUATION OF PREVENTION EFFORTS Evaluating prevention efforts may be challenging. In order to support evaluation efforts, the last section of this survey asks about your organization’s approach to evaluating your prevention work. 24. How do you know your prevention programs are successful at achieving the outcomes you want? How Your Agency Evaluates Its Prevention Efforts 25. During the past year, which of the following approaches has your organization used to evaluate its prevention work? ______ We have not evaluated our prevention work during the past year Who Lead Evaluation (select all that apply): ______ Our own staff led an evaluation ______ Someone outside our organization led an evaluation on a voluntary basis ______ We paid someone outside our organization to lead an evaluation ______ Our state coalition evaluated our work ______ Our RPE coordinating agency evaluated our work ______ Other (specify: ___________________________________________) Methods Used (select all that apply): ______ Survey (including pre-post test surveys) ______ Interviews ______ Focus Groups ______ Observations ______ Archival Data (e.g., sexual harassment complaints, police records, etc.) ______ Other (specify: ____________________________________________) 26. Which of the following types of outcomes do you measure when you evaluate your prevention programs? ______ Participants’ satisfaction with the program ______ Knowledge about sexual assault (e.g., definitions, facts, etc.) ______ Attitudes about rape (e.g., rape myth acceptance, etc.) ______ Intent or likelihood of behaving in certain ways (e.g., intervening as bystanders, committing acts of violence, etc.) ______ Actual behaviors (e.g., actual interventions as bystanders, perpetration, etc.) 27. There are many approaches to evaluating program outcomes. The list below names some tasks that are often completed during an evaluation. Please rate how confident you are in your agency’s ability to carry out these tasks. Very A Little Moderately Very Unsure Confident Confident Confident Develop logic models Define program goals and objectives Design an evaluation (e.g., figure out how to collect data) Develop or select a way to measure outcomes (e.g., surveys, interviews, focus groups, etc.) Analyze numerical/quantitative data Analyze open-ended/qualitative data Use evaluation findings to improve our work 28. Is there anything else the NSVRC needs to know about what is happening in your community, state or territory around prevention or issues you think need to be addressed at the national level? THANK YOU Thank you for taking the time to complete this survey. In order to provide you with the $25 stipend for completing this survey, the independent evaluator will need to have your agency’s name and mailing address. Reminder: The only person who knows you completed the survey and who will see your individual answers is the independent evaluator who has been contracted by the NSVRC to lead the evaluation. Neither the NSVRC staff, any state coalition staff, nor any funders will know which organizations completed the survey. If you wish to skip this question, you may do so. (However, in that case you will not receive the $25 stipend.) Agency Name: __________________________________________ Mailing Address: ________________________________________ ________________________________________ State/Territory: __________________ ZIP: __________________ If you have any questions about this survey you may contact: at the NSVRC: Jennifer Grove, 877-739-3895 x. 121, jgrove@nsvrc.org the evaluator: Stephanie Townsend, PhD, 585-690-9315, stephanie.townsend@earthlink.net Appendix D: Statistical Details Independent Samples t-Tests for Differences in Endorsement of Prevention Principles Between RPE Coordinators and Coalitions df t Multiple Strategies 46 -0.51 Multiple Settings 46 -0.01 Varied Methods 46 -0.65 Multiple Times 46 -0.86 Theory Driven 46 -2.59* Positive Relationships 46 -2.02 Timed 46 -1.70 Culturally Relevant 46 0.74 Evaluated 46 -2.31* Trained Staff 46 -1.40 Mobilize Community 46 -0.69 Oppression 45 1.21 Feminism 44 0.69 Independent Samples t-Tests for Differences in Endorsement of Prevention Principles By RPE Funding df t Multiple Strategies 58 -0.31 Multiple Settings 57 -0.91 Varied Methods 58 -1.45 Multiple Times 58 -1.53 Theory Driven 57 -2.46* Positive Relationships 58 -0.84 Timed 56 -0.88 Culturally Relevant 57 -0.24 Evaluated 58 -1.52 Trained Staff 58 -1.60 Mobilize Community 58 -1.12 Oppression 56 0.17 Feminism 56 -0.87 ANOVAs for Differences in Endorsement of Prevention Principles By Community Setting SS df MS F Multiple Strategies 1.37 2 0.68 1.53 Multiple Settings 2.21 2 1..10 2.25 Varied Methods 3.16 2 1.58 3.53* Multiple Times 0.83 2 0.41 0.81 Theory Driven 7.28 2 3.64 4.43* Positive Relationships 1.58 2 0.79 1.71 Timed 0.81 2 0.41 0.49 Culturally Relevant 1.91 2 0.96 2.05 Evaluated 2.73 2 1.36 2.07 Trained Staff 1.60 2 0.80 1.94 Mobilize Community 6.01 2 3.00 5.62* Oppression 3.84 2 1.92 2.64 Feminism 0.58 2 0.29 0.31 * P<.05 ANOVAs for Differences in Endorsement of Feminism SS df MS F SA vs. Dual Coalitions 0.84 2 0.42 0.32 RPE Coordinator Setting 1.96 2 0.98 0.89 Type of RCC 0.32 2 0.16 0.17 RPE Funding for RCC 0.67 1 0.67 0.75 RCC Community Setting 0.58 2 0.29 0.31 Correlations for Associations with Endorsement of Feminism State/Territory Local Fulltime Employees 0.19 -0.11 PT Employees 0.27 0.07 Multiple Strategies 0.18 0.01 Multiple Settings 0.02 0.05 Varied Methods -0.08 0.13 Multiple Times -0.02 0.13 Theory Driven -0.07 0.22 Positive Relationships 0.24 0.13 Timed 0.30* 0.19 Culturally Relevant 0.14 0.23 Evaluated 0.07 -0.04 Trained Staff -0.02 0.02 Mobilize Community -0.07 0.12 Oppression 0.35* 0.39* Paired Samples t-Tests for Changes in Beliefs About Prevention for RPE Coordinators and Coalitions df t Possible 45 6.78* Easy 45 6.03* Rewarding 45 4.00* Clear 44 7.69* Already Doing 44 6.29* Consistent 43 3.61* Complementary 43 5.58* Social Change 44 3.468 Community-Wide 45 4.01* Agency-Wide 43 4.848 Paired Samples t-Tests for Changes in Beliefs About Prevention for Local Programs df t Possible 67 -5.55* Easy 67 -4.82* Rewarding 67 -5.04* Clear 67 -7.24* Already Doing 66 -7.02* Consistent 64 -3.41* Complementary 66 -3.86* Social Change 66 -2.46* Community-Wide 67 -3.98* Agency-Wide 67 -3.83* Chi-Squared Analyses for Role Differentiation Between Coalitions and RPE Coordinators df Chi-Squared Information Re: Promising Practices 2 0.99 Recommending Curricula/Activities 3 3.12 Mandating Curricula/Activities 3 5.81 Statewide Prevention Initiatives 3 2.78 Training RCCS on Primary Prevention 3 4.91 Training Allied Organizations/Professionals 3 2.48 Providing Networking 3 3.60 Technical Assistance 3 1.34 Educating Legislators 3 11.39* Building Evaluation Capacity 2 0.75 Evaluating Local Initiatives 3 5.79 Disseminating Research to RCCs 2 1.07 Disseminating Research to Allied Organizations/Professionals 2 0.26 Conducting Research 3 5.15 Bringing Together Rape Prevention and Health Organizations 2 1.19 Bringing Together Rape Prevention and Other Allied 2 0.53 Organizations Working with Culturally Specific Programs 2 1.79 Chi-Squared Analyses for Activities for Different Community Settings df Chi-Squared General Rape Awareness Education 4 1.79 Bystander Empowerment 2 5.75 General Social skills Training 4 1.25 Gender Issues Training 4 3.14 Media Literacy Training 4 4.49 Anti-oppression Training 4 2.52 Mobilizing Men 4 7.77 Training Professionals to Do Primary Prevention 4 4.34 Changing Norms Campaigns for Prevention 4 6.42 Community Mobilization for Prevention 4 8.56 Coalition Building for Prevention 2 6.64* Public or Organizational Policy Advocacy 4 4.04 Systems and Organizational Change 4 3.39 Prevention Strategies for Specific Communities or Cultural Groups 4 2.65 Chi-Squared Analyses for Activities for RPE-Funded vs. Non-RPE Funded RCCs df Chi-Squared General Rape Awareness Education 2 2.02 Bystander Empowerment 1 3.83* General Social skills Training 2 1.20 Gender Issues Training 2 2.02 Media Literacy Training 2 2.32 Anti-oppression Training 2 1.18 Mobilizing Men 2 5.07 Training Professionals to Do Primary Prevention 2 3.60 Changing Norms Campaigns for Prevention 2 7.91* Community Mobilization for Prevention 2 8.51* Coalition Building for Prevention 1 6.84 Public or Organizational Policy Advocacy 2 0.01 Systems and Organizational Change 2 0.43 Prevention Strategies for Specific Communities or Cultural Groups 2 1.68 Factor Analysis on Barriers/Supports to Prevention Work (Principal Component Analysis with Varimax Rotation) Component 1 Component 2 Component 3 Component 4 Component 5 Component 6 Information .22 .13 -.01 .17 .76 .31 Ideas .14 .07 .09 .22 .80 -.13 Spanish .12 .07 .68 .32 .01 .16 Other Languages .09 -.18 .78 .26 -.11 .25 Oral .04 -.05 .74 .24 -.15 -.05 Cultural Materials .11 .13 .85 -.14 .15 .07 Cultural Skills .21 .24 .74 -.11 .36 -.014 Research .86 .17 -.02 .04 .24 .18 Research Skills .77 .06 .02 -.24 .13 -.01 Databases .74 .17 .22 .33 .05 -.04 Online Technology .74 .26 .28 .06 -.14 .27 Other Programs .60 .10 .18 .51 .20 -.09 Networking .57 .25 .20 .42 .25 -.18 Bridge .48 .26 .55 .14 .20 .03 Experts .81 .29 .15 .15 .06 -.08 CDC .19 -.05 .05 .75 .32 -.16 Allied .26 .49 .28 .47 .13 .04 Coalition/RPE -.06 .03 .22 .79 .05 .25 Evaluate .35 .64 .07 .06 .21 -.44 Retain .20 .75 .28 -.08 .03 -.23 Staff .05 .81 .02 .01 .09 .20 Agency .08 .86 01 .02 -.02 .12 Jobs .09 .23 .23 .04 .08 .73 Org. Leadership .29 .78 -.08 -.09 .03 .28 Funding .28 .70 -.00 .32 .07 -.03 Chi-Squared Analyses for Barriers Between Coalitions and RPE Coordinators df Chi-Squared Information 1 3.49 Culturally Specific 1 .05 Research 1 4.06* Networks 1 .01 Organizational Capacity 1 2.36 Chi-Squared Analyses for Barriers Between RPE-Funded and Non-RPE Funded RCCs df Chi-Squared Information 1 1.49 Culturally Specific 1 2.55 Research 1 2.17 Networks 1 1.79 Organizational Capacity 1 0.25 Chi-Squared Analyses for Barriers Between Community Settings df Chi-Squared Information 2 0.35 Culturally Specific 2 3.12 Research 2 3.17 Networks 2 3.66 Organizational Capacity 2 1.34 Chi-Squared Analyses for Specific Barriers Between Coalitions and RPE Coordinators df Chi-Squared Spanish 2 0.08 Other Languages 2 0.14 Oral 2 0.09 Cultural Materials 2 0.87 Cultural Skills 2 0.50 Evaluate 2 2.92 Retain Staff 2 4.72 Number Staff 2 3.43 Agency-Wide 2 6.65* Organizational Leadership 2 11.39* Funding 1 0.63 Chi-Squared Analyses for Specific Barriers Between Community Settings df Chi-Squared Spanish 4 4.27 Other Languages 4 5.37 Oral 4 0.93 Cultural Materials 4 5.51 Cultural Skills 4 3.11 Evaluate 4 1.59 Retain Staff 4 2.64 Number Staff 4 0.52 Agency-Wide 4 1.46 Organizational Leadership 4 1.76 Funding 4 6.32 Chi-Squared Analyses for Specific Barriers Between RPE-Funded and Non-RPE Funded RCCs df Chi-Squared Spanish 2 1.74 Other Languages 2 0.93 Oral 2 1.04 Cultural Materials 2 2.13 Cultural Skills 2 4.21 Evaluate 2 4.34 Retain Staff 2 8.64* Number Staff 2 5.31 Agency-Wide 2 1.09 Organizational Leadership 2 1.58 Funding 2 3.34 Evaluator’s Background Stephanie Townsend, PhD, has worked in the movement to end sexual violence as both a practitioner and researcher. She began by working for community-based rape crisis and prevention programs in Michigan, California and Texas. During that time she also served on the boards of directors of the National Coalition Against Sexual Assault, the California Coalition Against Sexual Assault, and on the advisory board of the Texas Association Against Sexual Assault. She completed her doctoral work in community psychology at the University of Illinois at Chicago. Her research has focused on community-based rape prevention programs and Sexual Assault Nurse Examiner programs. She has conducted global, national, state, and local research and evaluation projects. She is a member of the American Evaluation Association, American Psychological Association, Society for the Psychology of Women, and Society for Community Research and Action.