Service To Science Project / Evaluation Design
SIGNALS RECIEVES FEDERAL PROJECT AWARD
-KENTUCKY BASED PROGRAM WINS-
SOUTHEAST CAPT REGION
Pilot Evaluation Capacity Building for Evidence-based Interventions
The SIGNALS Program (Setting Important Goals Now Against Life-threatening Substances) of The Kentucky United Methodist Homes for Children & Youth. received notification of a project award supported by the U.S. Department of Health and Human Services (DHHS), The Substance Abuse and Mental Health Services Administration (SAMHSA) and the Center for Substance Abuse Prevention (CSAP). The project award is a mini-subcontract. It provides $30,000. to SIGNALS to enhance the evaluation capability of the program.
The SIGNALS curriculum met all the criteria— an innovative, locally developed, evidence-based intervention tool. Published in 2003, the curriculum is taught in 19 states, and over 70 Kentucky organizations, including public and private schools, churches and treatment centers. The SIGNALS Program is based on a teen substance abuse prevention, intervention and education curriculum written by David Lane Stonecypher, Director of Substance Abuse Prevention Services at The Methodist Children’s Home. The program was nominated and supported for the award by the Kentucky Department of Mental Health and Mental Retardation and the Division of Substance Abuse.
The SIGNALS program was the only Kentucky-based program to be awarded this year. SIGNALS is one of two programs selected to represent the United States Southeast region, which includes nine southern states and three districts, including Washington D.C., the U.S. Virgin Isles and Puerto Rico. SAMHSA and CSAP fund 25 programs nationwide with the mini-subcontracts every year. They developed the mini-subcontract award as a component of the larger Service to Science Initiative. Programs must submit a response to an RFP (request for proposal) and advance through a two-tier review panel of nationally recognized experts in the field. Awardees attend one of CSAP’s Centers for the Application of Prevention Technologies to complete the Service to Science Academy.
Enhancement Evaluation Plan / Service to Science Project
The purpose of the evaluation enhancement plan is four fold. Information will be gathered and analyzed to first gain insight in how the design of the programs activities can be improved to enhance expected changes. Next, information will be collected and examined to determine whether any change practices are needed to improve the quality, effectiveness, and efficiency of program activities.
Furthermore, the proposed evaluation plan will be implemented to assess the effects of program interventions on expected outcomes by examining the relationship between program activities and observed consequences. Finally, careful consideration will be given to monitoring the impact on stakeholders of the processes and procedures required to conduct an evaluation.
It is assumed that the overriding focal point of this evaluation enhancement plan is to improve the evidence base of credible and reliable information that can be used by the various stakeholders to determine the merit, worth, and significance of the SIGNALS program as a viable and recommendable prevention resource. This proposed evaluation enhancement plan primarily focuses on improving the process used to assess effects of the program interventions on expected outcomes and their attribution, thereby enhancing the documentation of evidence needed to determine and substantiate program efficacy.
Specifically, the evaluation enhancement plan proposed is designed to generate an accurate and repeatable evidence base that will enable stakeholders to address and answer six questions considered crucial in determining the quality, value, and significance of the SIGNALS curriculum.
These questions are; to what extent does participation in the SIGNALS program:
Enhance a youth’s ability to accurately assess their risk level for developing an ATOD use problem?
Lower the youth’s resistance to and increase the acceptance of a non-use prevention message?
Reduce the youth’s risk level for developing an ATOD problem?
Add to the youth’s development of self preservation awareness and lower risk perception deficit?
Improve a youth’s accurate knowledge base about the harm of using ATOD?
Reduce a youth’s defensive postures and acceptance of environmental myths?
These questions will help answer the overriding question:
To what extent does equipping youth with an accurate assessment of their risk level combined with enhanced self- preservation awareness increase a youth’s acceptance of a non-use prevention message and create a sustained non-use of ATOD?
In addition, the extent to which grade level, sex, prior risk level, and/or stage of change level will impact the answers to the above listed questions will be examined.
ALSO SEE "SECULAR AND SCIENCE-BASED CURRICULUM"
The SIGNALS Service to Science Project was officially completed with the submission of the Final Report to Washington, D.C. on May 30th 2008.
The SIGNALS website will create a new section to describe the outcomes of the project. Until then, here is a PDF version of the Final Report. Final S2S Report PDF
