On the Outside Looking In: The G/T Program Evaluation Process

Every journey begins with a first step. In order to take that step, a destination is determined. Deciding the journey’s destination relies on prior travel and a desire for a new direction to explore. A G/T program services evaluation journey looks at both the places traveled and any new destination that leads to student achievement. The journey of gifted services is one that relies on past experiences, current needs, and best practices for the students and educators in the district or on a campus (Boswell, 2018).

When we think of the final destination of a G/T program as implementing successful gifted services in schools, we can easily imagine adventurous programs where robust and new ideas inspire students led by innovative visionaries striving to think “outside of the box.” Although gifted programs led by such adventurous educators and innovative leaders stand out, we must understand that these trailblazers may serve as an example for all involved in building paths to outstanding services. Sometimes the most important thinking and difficult work comes from “inside of the box,” yet it helps when we have eyes looking from the outside to help our journey toward our destination. Evaluations of G/T program services can make that work easier. The Texas State Plan for the Education of Gifted/Talented Students (State Plan; Texas Education Agency [TEA], 2019) provides the roadmap for the journey.

Texas State Plan for the Education of Gifted/Talented Students

Administrators of gifted services should reflect on how to accomplish the following directives from the State Plan:

Family/Community Involvement

6.8 The effectiveness of gifted/talented services is evaluated annually, shared with the board of trustees, and the data is used to modify and update district and campus improvement plans. Parents are included in the evaluation process, and the outcomes and findings of the evaluation are shared with parents (TEC §§11.251–11.253). (p. 12)

In addition, the State Plan’s section “Fidelity of Services” requires the following:

• 1.5 Annual evaluation activities are conducted for the purpose of continued service development.
• 1.5.1 Ongoing formative and summative evaluation strategies, based on quantitative and qualitative data, are reviewed by the school board and used for substantive program improvement and development.

• 1.6 Long-range evaluation of services is based on evidence obtained through gifted/talented-appropriate performance measures such as those provided through the Texas Performance Standards Project (TPSP).
• 1.7 The development and delivery of curriculum for gifted/talented students is monitored regularly by trained administrators.
• 1.8 District guidelines for evaluation of resources used to serve gifted/talented students are established and used in selecting materials that are appropriate for differentiated learning.
• 1.9 Curriculum for gifted/talented students is modified based on annual evaluations. (p. 2)

Although the State Plan directs that there should be an annual evaluation with data to meet these stipulations, G/T administrators may approach this effort in a variety of ways. A good place to start is with a complete evaluation of services for the six areas in the State Plan: Fidelity of Services, Student Assessment, Service Design, Curriculum and Instruction, Professional Learning, and Family/Community Involvement. Options to accomplish this evaluation include:

• internal or external management,
• students’ services to determine student growth, and/or
• oversight managed by the coordinator or by district administrators.

As long as results are used for a district and campus improvement plan’s modification and parents are a part of the process, the district may approach evaluation as best fits local needs.

With State Plan requirements in mind, what should coordinators look for or expect to happen from an evaluation or audit? Should the evaluation be internal or external? What should preparation look like before the evaluation/audit begins? What outcomes will facilitate program growth? What tips may facilitate efforts toward a positive outcome for all involved, including students, educators, and parents?

Evaluation or Audit

On more than one occasion, we have been asked to complete an audit of gifted services. On more than one occasion, we have been asked to complete an evaluation of gifted services. What is the difference between the two? When probed for an expected outcome, there was no difference in expectations.

What is the difference? According to Ian Clark (2014) in The Atlas of Public Management, Clark cited definitions from Merriam-Webster’s Online Dictionary to clarify the difference: (1) An internal audit is a formal examination of an organization’s or individual’s accounts or financial situation, and (2) an evaluation is a way to determine the significance, worth, or condition of usually by careful appraisal and study.

Clark continued by saying that the goal of an audit is accountability and improvement, while the goal of an evaluation is improvement and accountability. So, the difference? Performance measures show a distinction between the two terms. An audit looks to determine if a performance measurement is in place, while the evaluation asks what the performance measures tell us and how they may be improved.

From these corporate examples, it’s clear that a gifted evaluation, as described in the State Plan, leans toward evaluation more than audit. And, as such, consideration of program services for the gifted favors outcomes of the effort and performance measures to guide the evaluator.

Internal or External

Be it termed an audit or an evaluation of a district’s G/T program services, instructional leaders must first decide whether an internal or external review of the system best meets the needs of gifted students.

Internal reviews are typically conducted by a committee and give an overview of all aspects of the program services or to a specific area of concern. The findings are then communicated to interested parties and an action plan is developed.

Internal reviews can have limitations. As they evolve and mature, our programs become filled with routines and traditions. We may not know why we do what we do in our program but rely on the comfort of past processes and therefore delay acting on necessary changes to push services forward. Because we are so close to the situation, changes we may need to make cause us to become paralyzed with fear of making a wrong decision or unable to see other possible roads.

Neumeister and Burney (2012) said that an external review is best practice. They reported that a formal [external] program evaluation offers the following:

• an objective view from an expert in gifted education,
• identification of strengths of the program,
• identification of challenges that need to be faced,
• new directions to consider,
• next steps in improving a program, and
• assistance in public perceptions of credibility (p. 82).

Their findings align with the outcomes posited by the State Plan. An objective evaluation by an expert in gifted education offers credibility to strengths and recommended areas for improvement. Ultimately, the expert’s proposals facilitate modifications for district and campus improvement plans.

Preparations

It is not uncommon for coordinators to be uncomfortable as a program evaluation approaches. Fear of the unknown is common. The best approach is to be flexible and willing to accept possible new and viable ideas. Consider that it could be worse for the district and, most importantly, for the gifted students served if you do nothing. When determining best practices for student growth is stated as one goal of the evaluation, not as a “gotcha” toward individual teachers, coordinators, or administrators, the focus moves from personal to policy.

For outcomes to be realized, the first step is to state guiding questions. What outcomes does the district expect from the evaluation? Questions are best limited to no more than three. For example, the following questions may guide outcomes related to program design and curriculum and instruction:

1. How does Good Hope ISD follow best practices in program design and curriculum and instruction to meet the needs of gifted and talented students?
2. How could G/T services in Good Hope ISD be redesigned to better serve the academic, social, and emotional needs gifted and talented students?

Other guiding questions to focus on assessment for identification and student achievement might be these:

1. How does Good Hope ISD follow best practices in assessment to reflect district demographics in G/T services?
2. How do identified G/T students in Good Hope ISD show growth in student achievement?
3. How could G/T services in Good Hope ISD be redesigned to better serve the academic, social, and emotional needs gifted and talented students?

For an in-depth look at and full description of the evaluation process, access the following three in-depth articles about the evaluation process, including questions that guide it:

• The Evaluation Journey
• How to Evaluate Your School or District G/T Program
• What the Research Says About Evaluating Gifted Programs

Kaul et al. (2018) provided three charts that offer examples of planning, data collection procedures, and an action plan. Readers may also refer to the following TEMPO+ resources on evaluation, “Conducting an In-Depth Evaluation of a Gifted Program” and “Using the Texas State Plan to Evaluate Gifted and Talented Programs.”

Once the evaluation outcome(s) and guiding questions are developed, administrators begin the process of finding an external auditor. This process could involve recommendations from the Texas Association for the Gifted and Talented (TAGT), the state department of education, or colleagues in similar-sized districts who have completed an external evaluation.

Although Neumeister and Burney (2012) recommended an external evaluation, a preliminary internal evaluation could be an ingredient of a complete evaluation. If the district uses annual surveys and interviews as stated in the State Plan, data from the stakeholders’ surveys (those of students, parents, and educators) facilitate the work of the external evaluator. In addition, insights from practitioners and community members through focus groups, classroom observation, and student growth data add to the evaluator’s knowledge base.

Other guidance provided to the evaluator might include information from a local website or a handbook of gifted services. These sources may include documents defining the district’s plan for student assessment and program design to meet local needs, such as curriculum documents or scope and sequence. Current campus and district improvement plans are essential for final recommendations that complete an evaluation.

Outcomes

The time, energy, and hours invested in an evaluation of a district’s G/T program services become worthless if no action is taken on the recommendations and new ideas generated by the process. A district will never know whether new ideas will work unless there is a willingness to take risks and implement performance measures.

Before performance measures can be determined, the outcome of the evaluation must be stated. Outcomes are related to the original purpose of the evaluation as stated in the guiding questions. For some districts, a 5-year plan is in place that is ready for evaluation of success with services. For others, there may be a need to determine success of services considering student growth, meeting student assessment goals, or demographic goals for assessment and services.

Ultimately, the evaluation contributes to the impact of the program toward continuous improvement (Swenson & Lord, 2013). Johnsen (2010) stated, “The number of practices is not as important as whether the practice is effective with students” (p. 5).

Tips

The following tips come from working with districts across the state. Whether the evaluation is a desk review or a long-term evaluation, focusing on the outcome ensures that valuable information to improve services for gifted learners follows:

• Most of all, ensure that all parties approach the evaluation with an open mind. The evaluation’s purpose is to improve services for students.
• Guiding questions are essential. A haphazard evaluation, whether internal or external, yields no useful results.
• Welcome the external evaluator. Facilitate the evaluator’s efforts by having documents and personnel ready and welcoming. Provide as much information as possible to the external evaluator.
• Open classrooms for observations.
• Offer time to interview school personnel, parents, and students.
• Include the school’s superintendent and Board of Trustees in the process. Without their support, including monetary support, any changes will be hindered or obstructed.

District G/T leaders can now consider the answers to their guiding questions, ponder recommendations, and then set out the best map to ensure reaching a program service option that meets the school’s goals, prioritizes gifted students’ needs, and matches the styles of teaching and leadership in the district.

References

Boswell, C. (2018). The evaluation journey. TEMPO, 34(1), 33.

Clark, I. (2014). Comparing audit and evaluation. Atlas of Public Management. http://www.atlas101.ca/pm/concepts/comparing-audit-and-evaluation

Johnsen, S. (2010). Principles underlying the 2010 programming standards in gifted education. Gifted Child Today, 34(2), 5. https://doi.org/10.1177/107621751103400202

Kaul, C., Davis, B., & Johnsen, S. (2018). How to evaluate your school or district G/T program. TEMPO, 34(1), 6–17, 39, 40.

Neumeister, K. S., & Burney, V. H. (2012). Gifted program evaluation: A handbook for administrators and coordinators. Prufrock Press.

Swenson, J. D., & Lord, E. W. (2013). Harnessing and guiding the power of policy. Examples form one state’s experiences. Journal for the Education of the Gifted, 36(2), 198–219. https://doi.org/10.1177/0162353213480434

Texas Education Agency. (2019). Texas state plan for the education of gifted/talented students. https://tea.texas.gov/Academics/Special_Student_Populations/Gifted_and_Talented_Education/Gifted_Talented_Education

On the Outside Looking In: The G/T Program Evaluation Process PDF