NCOA Best Practices Report

Building on Best Practices:

Consensus Report on Physical Activity Programming

 
Background

In recent years, evidence-based physical activity programs developed and tested in academic research settings have become the gold standard in producing measurable health benefits and positive outcomes for older adults. An important national health promotion priority is to encourage the adoption of evidence-based programs and facilitate the use of best practice principles at the community level to improve quality and ensure the safety and effectiveness of physical activity programming. 

To that end, the National Council on the Aging (NCOA) and other health promotion organizations are searching for the ways and means to deliver information about evidence-based programs and best practice guidelines to program providers.  Providing high quality physical activity programming to older adults is becoming increasingly important: Funders often require evidence of effectiveness for continued financial support, and program providers are looking for programs with proven effectiveness in which to direct scarce resources. In addition, older adults are now starting to look for evidence-based, proven programs in their communities.

In September 2007, NCOA, the National Blueprint Office, and Active for Life convened a meeting entitled “Building on Best Practices: Physical Activity Programming in the Aging Network” to address issues related to widely disseminating information on best practices and evidence-based programs to community organizations that serve older adults.  Nearly 30 experts in physical activity and aging and public health, and representatives from state governments and funding organizations shared in discussions at the University of Illinois at Urbana-Champaign to identify strategies.  The goals of the meeting specifically were to (1) identify the tools and resources that communities would need to implement evidence-based programs; (2) review existing principles of best practices and determine what is known about the feasibility and adaptability of these principles; and (3) explore ways to evaluate the effectiveness of physical activity programs, as well as identify resources that organizations can use to support these efforts.

This report relates the discourse, recommendations, and outcomes of the national meeting.  Its intent is to provide NCOA and other national organizations with recommendations on disseminating information about evidence-based programs and best practice principles, and providing continued support for community organizations in offering high quality, effective physical activity programming.
  
Selecting Evidence-Based Physical Activity Programs

What is an evidence-based program?  Even among experts in the field, there is no standardized definition.  However, meeting participants provided a list of criteria that would distinguish this type of program from other offerings.  They agreed that evidence-based programs are usually (but not always) those that have been tested in a randomized clinical trial with statistically significant benefits, which can be duplicated in multiple settings and with diverse populations.  Additionally, trial outcomes have been published in a peer-reviewed academic journal, and the program provides reproducible materials such as manuals or toolkits.  Evidence-based programs generally adhere to widely accepted ACSM/AHA physical activity recommendations and guidelines and almost always include a rigorous evaluation component.  Validity and quality are important criteria in the development of evidence-based programs, but affordability and sustainability are critical considerations at the local level.

 Program Access and Support

In order for community organizations to select evidence-based programs for their members, they need basic information about the types of programs available and assistance in choosing the most appropriate programs to meet their goals.  The experts recommended that a listing of evidence-based programs should be offered, and that information should be available through the Internet on several national organization websites and disseminated to regional aging network offices and agencies.  Other suggestions included specifying the resources and costs that are required for each program so providers can determine the cost/benefit ratio based on the number of participating members.  Materials should be user-friendly and provided in various formats, such as issue briefs and fact sheets, which assist in selecting and promoting programs and sharing information with local leadership boards.

Once organizations learn about the offerings available, they will need technical assistance, training, and education in implementing programs.  Specifically, they may need assistance in choosing programs, adapting programs while maintaining program integrity, determining outcomes, and obtaining funding.  The recommendations included providing a simple checklist to summarize the evidence-based programs available in terms of costs, target populations, and other criteria, and providing promotional materials to recruit participants.

Locally, university partnerships and assistance from Area Agencies on Aging are invaluable for implementing and evaluating programs. Graduate students and faculty can train local providers on how to evaluate and improve programming and assist with problem-solving.  The experts also recommended linking new organizations with those that have previously used a selected evidence-based program, and linking organizations with the program manager for technical assistance, perhaps via list serves.

Organizations that could be charged with the task of providing education and assistance included national organizations related to physical activity and aging (Administration on Aging, AARP, NCOA, International Council on Active Aging, etc.), trade associations, Centers for Medicare and Medicaid Services, Association of Chambers of Commerce, and the National Governors Association.  At the local level, city/county government and health officials, Cooperative Extension offices, and community coalitions could provide needed support.

The experts debated the most efficient ways that information about evidence-based programs could be disseminated to local organizations. A few suggestions included simple paper-based resources, such as checklists and fact sheets; readiness assessment tools; hotlines or list serves to provide leadership and problem-solving; and web-based PowerPoint presentations to view and use as needed.  They also recommended that NCOA develop issue briefs on why evidence-based programs are superior to other types of programs, and suggestions, tools, and resources for selecting programs.  National organizations, such as the National Blueprint, would assist in disseminating issue briefs and other materials.

Best Practice Principles

Community organizations that lack the desire or funding to adopt packaged evidence-based programs may consider using principles of best practices to improve their existing programs.  They may be motivated to improve programs in order to obtain or sustain funding, increase participation, and receive better health outcomes.

Many of the same national organizations that disseminate information about evidence-based programs could also promote best practices in communities.  Program providers could also learn about best practices through publications, professional associations, web-based discussion groups, and peer support.  They will likely need various types of technical assistance.  Community program providers typically have a paucity of financial resources and knowledge about best practices and how to implement these practices into their programs.

Manuals and toolkits are useful tools for improving programs.  The experts also suggested developing trainings, videos, and photos depicting specific best practice options.  Checklists that innumerate best practices allow providers to scan information and assess activities and options that they currently offer.  Checklists should be user-friendly and written in such a way that the concepts are easy for anyone to understand and are adaptable for various populations of older adults.  A few experts recommended that checklists should only be provided with personalized follow-up consultation. 

Non-traditional Settings and Populations

In order to improve physical activity programs through the use of best practices in non-traditional settings and diverse populations, it is particularly important for providers to determine what their clients expect from a physical activity program.  Community providers should use both trusted professionals and lay leaders to instruct programs.  Providers could increase the acceptability and utilization of programs by cultivating relationships with community leaders from various cultures to identify needs and goals.  Programs should always be culturally competent to increase effectiveness.

Evaluation

Although evaluation is essential to assess whether a program is effective and reaches its intended population, evaluation studies are rarely used in community-based programs for many reasons: for example, a lack of experience using evaluation tools, a lack of staff time for such efforts, a lack of funding to support evaluation, and a failure to understand the importance of obtaining outcome measures.

In fact, all parties benefit when evaluation tools are used. Program participants can use evaluation outcomes to select a program that works and obtain feedback about their own performance.  Program providers benefit by learning how to improve their programs, by attracting new clients through promotion efforts that clearly enumerate positive outcomes, and by obtaining new or sustained funding.  Finally, funding organizations often require evaluation outcomes so they can ensure that their financial support is being used appropriately to meet the organization’s underlying goals. Funders may be interested in the changing health and/or quality of life measures of participants at one point in time, or may desire to understand the long-term impact the programs have in the community.  In addition, state departments use evaluation outcomes to obtain resources through legislation and from federal funding agencies.

Evaluation Philosophy

Although the experts agreed that evaluation efforts should be encouraged at the community level, they noted that poor data is worse than no data at all.  Therefore, they recommended that providers use simple outcome measures with clients to increase data validity.  Measures might include program demographics, attendance and attrition rates, self-rated health questionnaires, and participant testimonials to determine if the program met the individual’s goals.  Comparing qualitative assessment data at two points in time can provide meaningful outcomes, particularly for quality improvement.  Simple evaluation measures can provide feedback to participants so they can chart their own progress.  One suggestion was the three-minute Centers for Disease Control and Prevention’s Healthy Days survey, used to measure quality-of-life issues (available at http://www.cdc.gov/hrqol/methods.htm#healthy).  Clients benefit when they have the opportunity to provide testimonials or share suggestions for program improvement.

Ideally, evaluation should be a part of program planning, and evaluation goals and expectations should be clearly stated before the program begins. Additional suggestions include ensuring that evaluation measures are appropriate for a given population, selecting measures that will provide meaningful outcomes for the organization and individuals, and training staff to use measurement tools.  Special events could be used to assist with the efficient scheduling of measurement testing.  For example, some organizations use the Senior Fitness Test annually during participants’ birthday months.  Additionally, a reunion event could provide an opportunity for post-program testing.

Technical Support

Undoubtedly, organizations will need technical assistance when evaluation measurement is a new concept or in using new measurement tools. Administrators and staff may require training on evaluation measures to learn about the benefits and how to use surveys and tests properly.  Training could take place at regional and annual health-related conferences, and a train-the-trainer system could be used to educate staff.  The experts suggested that evidence-based program manuals and materials should include evaluation tools, including spreadsheets and reports, and possibly, offer direct technical assistance (via e-mail, phone) on using these tools.  Another recommendation is to develop a menu of evaluation choices and contact information for technical support.

Organizations would also benefit from receiving information on ways to present evaluation outcomes to participants to help them to understand the outcome implications.  They also need information on communicating the results to funders in a compelling way. An NCOA issue brief would be useful in disseminating this information.  NCOA may also disseminate information on evaluation issues to other health-related national associations to pass on to regional/local offices.

Partnerships are invaluable for sharing information and resource tools, particularly in obtaining information on program evaluation.  Sources of information may include local university faculty, government offices at the local, state, and federal level, local Area Agencies on Aging, health departments, regional health clinics, local foundations, and national organizations, possible candidate organizations include;  the United Way, AARP, ICAA, and the Administration on Aging. The task of finding technical support would be greatly enhanced by having an online database or printed directory of professionals with various expertise to conduct trainings and provide other types of technical support for educating organizations on program evaluation and measurement.  

Conclusion

Evidence-based physical activity programs provide solid evidence of effectiveness, and best practices have been identified as the key components of any exceptional program.  These programs and practices were developed, tested, and reviewed with the goal of improving interventions provided to older adults for greater health benefits.  As it will become increasingly important for program providers to offer effective programs through the adoption of evidence-based programs and best practices, they will need support in program delivery.  National organizations that promote physically active lifestyles are in a unique position to disseminate these useful resources from the top experts in the field to program providers at the community level.

Workshop Participants

Terry Bazzarre             Robert Wood Johnson Foundation

Basia Belza                 University of Washington

David Buchner              Centers for Disease Control and Prevention

Wojtek Chodzko-Zajko  University of Illinois at Urbana-Champaign

Pat Cohen                    Illinois Adult Day Service Association

Shelly Ebbert                Illinois Department on Aging

Ron Ford                      Help at Home/Oxford HealthCare

Ronnie Gipson              Brazos Valley Council of Governments, TX

Don Grantt                    Administration on Aging

Sue Hughes                  University of Illinois at Chicago

Karen Lewis                  Comprehensive Health Education Foundation

Cathy Liles                    Texas A & M University

Leticia Malavasi              University of Illinois at Urbana-Champaign

Paula Martin                   Clark-Lindsey Village, Urbana, IL

Michael O’Donnell           East Central IL Area Agency on Aging

Jean O’Leary                   Madison School and Community Recreation

Marcia Ory                      Texas A & M University

Eunyoung Park                University of Illinois at Urbana-Champaign

Debra Reardanz               Clark-Lindsey Village, Urbana, IL

Amy Reeser                     Macon County Health Department, IL

Serena Sanker                  National Council on Aging

Lisa Sheppard                   University of Illinois at Urbana-Champaign

June Simmons                  Partners in Care Foundation

Susan Snyder                   Senior Services of Seattle/King County

Kathy Sykes                     Environmental Protection Agency

Jocelyn Tobnick                OASIS Institute

Nancy Whitelaw                 National Council on Aging

Lan-Jun Zhou                     University of Illinois at Urbana-Champaign