Q + A's on DoDEA's Program Evaluation:
Question #1: What is "Program Evaluation"?
Response:
Program evaluation is a structured method for determining the merit or value of an educational program. This format provides a comparative analysis of a program's expected end-goals with the actual programmatic results. It collects and analyzes the data necessary for purposeful data-driven decision-making and policy formation. It assesses student achievement and growth and allows for the monitoring of the use of public funds. Its ultimate goal is to improve educational materials and programs for students and associated stakeholders.
Question #2: What are the primary pieces of a program evaluation?
Response:
- Program Plan - developed during the planning and initiation of a program. It specifies the end goals and anticipated milestones, the data to be collected to illustrate progress, and the decisions that need to be made based on those data.
- Implementation of Program - accompanied by formative assessments and evaluations. As the program is implemented, cost benefit analyses are developed, implementation and modification timelines are established, analyses of the data are conducted, fidelity of the program implementation is determined, and the next actions are decided. Those next actions are: continuation, modification or termination of the program.
- Evaluation of Program - summative piece includes the data developed during implementation and recommendations on next steps.
- Adjustments and/or Elimination of Program - decisions with support on the next steps.
Question #3: Why does DoDEA conduct program evaluations?
Response:
DoDEA conducts program evaluations:
- to collect, analyze, and interpret data which can be used to develop and support decisions on future actions.
- to integrate new information and research into curricular programs on a regular planned basis.
- as part of our commitment to continuous improvement - to meet the changing needs of our students and communities and to document gains in student achievement.
Question # 4: What types of program evaluations does DoDEA conduct?
Response:
There are four general types of program evaluations:
- Formal Curricular Program Evaluations - these are conducted after formal implementation of curricular initiatives over a time frame of several (4-6) years. Example: Mathematics
- Issue-driven Program Evaluations - issues are identified by the Director or Deputy Director and the evaluations address or investigate the issues from a systemic perspective. Example: High School Counseling
- Program Evaluations done as part of a Program Objectives Memorandum (POM) or Special Initiative - these are specified within the formal agreement governing the development and implementation of the program. Example: Special Education Initiative
- Annual program reviews -conducted yearly on all educational programs. They specifically look at an educational program over the school year in terms of the actions taken and school-year goals achieved. Recommendations from these reviews guide actions to be implemented in the following school year. Example: Visual and Performing Arts Program Review for SY08-09.
Question #5: What is the process for conducting a program evaluation?
Response:
There is a standard template that is used as a guide for all four types of program evaluations.
Question #6: How are results from program evaluations used?
Response:
Analyses and recommendations generally are used to assist and support decision-making related to continuation, modification or elimination of programs.
Question #7: Who conducts the program evaluations?
Response:
HQ Program Coordinators or Branch Chiefs lead program evaluations and commonly include select area personnel as needed and input from the field.
Question #8: What is the connection between program evaluations and Task Groups?
Response:
Task groups provide a systemic review of a program and the issues facing it and are used to develop consensus recommendations. A task group's charge is provided by the Director and the task groups use the program evaluation template to guide their work.
Question #9: How are the results of a program evaluation shared?
Response:
DoDEA provides full reports to Senior Leadership and executive summaries to DoD committees such as Dependents Education Council (DEC) and Advisory Council for Dependents Education (ACDE).
Question #10: What DoDEA programs are currently in the evaluation process?
Response:
- Assessment
- K-12 Mathematics
- Professional Technical Studies
- PK-12 Literacy
Special Education Initiative will provide its summative evaluation at the end of this calendar year. Programs that developed their evaluation plans in SY 08-09:
- Virtual School
- Professional Development
- Partnership
